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T.  OESCPTPr  iri^'  OF  OFCISIOiN  AM^LrSlS 


Durina  tiTes  of  crisis  on  war/  the^e  is  orohablv  no 
field  of  enrieavor  like  rriilitarv  and  strategic  command  and 
control  wHere  decisions  are  made  cjuicklvf  but  under 
conaitions  of  risk  and  extreme  uncertainty.  This  is 
especially  tru^  since  acct.ira^'e  inforn^ation  on  all  ascects  of 
a  crcolem  are  almost  always  concealed  from  decision  makers. 
Every  military  force  tries  to  conceal  itself  and  r s<(^s  every 
attemot  to  deceive  the  ooposinq  decision  maker  as  to  its 
intentions.  facts  surround inq  the  tactical  situation  are 
elusive/  especially  those  concerning  the  enef^y.  These  tyoes 
of  decisions/  ♦'hen/  are  difficult  and  freauently  would  seem 
to  defy  a  systematic  decision  makino  process  (Decart'^ent  of 
the  Army/  1^8  0/  o,  3-2),  At  the  same  time*  there  is  no 
arena  in  which  bad  decisions  have  more  traqic  results.  The 
cost  of  making  errors  can  orow  exoonentially  mainly  due  to 
the  complex  r e 1  a t i onsn i ps  and  resulting  chain  reactions/  or 
the  errors  could  cancel  themselves  out  and  the  costs  not  Pe 
imrnediately  known  (Turban  and  N'eredith,  1^77/  o.   U), 

Conseai^iently/  while  decision  makers  must  recognize  that 
orderly/  rational  decision  makina  procedures  are  difficult/ 
they  must  at  the  same  time  constantlv  strive  to  base 
decisions  on  the  most  rational  thoucjht  process  possible.  To 
reduce  error  prooabilitv/  and  for  the  sake  of  survival* 
decision  makers  must  oecome  more  sophisticated   (Turban   ana 


'''eredith,  \^1 1 ,  o.  5).  They  must  learn  to  utilize  new 
tools  and  techninues  that  are  heina  developed.  '^io  one  could 
inr!;^'4ine  a  successtui  suraeon  utilising  equioTient  ano 
orocedures  froTi  the  turn  of  tne  century.  Yet/  in  decision 
•na<inQ»  you  can  still  find  aecision  makers  using  the  tools 
and  techniaues  of  that  tine. 

Military  and  strateoic  decisions  are  not  only  Tiade 
under  conditions  of  uncertainty  (where  not  all  facts  are 
av/ailanle)  and  stress^  they  ^re  also  rarely  considered 
^inal.  The  ev^r  changing  situation  brings  with  it  the 
regui^e■nen^  to  continuously  revise. acoraisalSf  estinr>ates/ 
ano  oerhaos  decisions.  Tnis  occurs  since  decision  making  is 
based  on  t^e  future  but  is  deoendent  on  the  oast.  For 
yearsf  manaqors  have  considered  decision  making  to  he  a  Dure 
art  01"  talent  which  is  acquired  over  a  long  oeriod  of  time 
through  exoerience  or  trial  and  error.  It  has  been 
consioered  an  art  recause  a  wide  variety  of  individual 
styles  can  oe  useq  in  aooroachino  and  successfullv  solving 
tne  same  tvoe  of  or'^hlpms.  One  woul.^  r,  ori^ally  base  these 
styles  on  creativity/  judgm, ent/  intuificn/  and  experience 
rather  than  some  sort  of  systematic  methoa  (Turt)an  and 
^^ereditn,  1977,  n.   5). 

■  Decision  making  as  a  discipline  han  its  origins  in 
operational  analysis  techniques  beqinninq  in  I'lorld  tiac  II 
(/nlliams/  1*^^78,  o.  12),  These  techniques  were  tyoically 
acplied  to  special  tyoes  of  clfar-cut,  reoetitive  problems/ 
such  as  those  of  systematic  search  and  resource  allocation. 


Sincp  the  I'^bO'Sf  However/  a  more  qener^l  tecHnoloqy 
has  eTierqeo  tor  incosinn  loaical  structure  en  the  reasoning 
thst  underlies  any  sn=>cific  decision.  Fiis  technolooy  is 
decision  analysis  (Barclay  et  a1.»  \9  7  7 ,  p.  iv).  Since 
1°70,  there  has  been  a  Tiajor  effort  by  defense  agencies  to 
adoot  this  technoloqv  to  their  dav-to-day  decision  making, 
f^any  have  found  it  a  way  to  make  better^  more  defensible 
decisions. 

Decision  analysis  is  a  quantitative  method  which 
oermits  the  svstematic  evaluation  of  the  costs  or  benefits 
accruim  from  courses  o^  action  that  might  be  tai<en  in  a 
decision  oroblem  (Barclay  et  a).,  1^77/  d.  vi).  The  method 
includes  the  ioentification  of  the  alternative  choices 
involved,  the  assignment  of  values  (costs/ r-)enefits)  for 
Dossible  outcomes,  ancj  the  expression  of  the  orobahility  of 
those  outcomes  occurring.  Once  this  is  done,  the  orobable 
aain  or  loss  associated  with  each  alternative  can  be 
determined  bv  systematically  combinino  the  probabilities  and 
values. 

In  addition  to  the  primary  role  of  decision  analysis  as 
a  method  for  the  loaical  solution  of  complex  decision 
problems,  it  also  has  several  additional  advantages  as  well. 
The  formal  structure  of  decision  analysis  insures  all  the 
elements,  their  relationships,  and  their  associated  weights 
have  heen  consioereo  in  the  decision  problem.  The  model  of 
the  decision  problem  can  serve  an  important  role  in 
facilitatina  communications  between  those   involved   in   the 


decision  orccess.  '^Iso  it  is  very  easy  to  identify  the 
areas  of  disaareeT'enf,  their  relative  inportancef  and  if 
they  ac*:u3lly  have  any  n^ateria!  iTtoact  on  the  indicated 
decision.  Firi3|]y,  when  changes  occur  in  the  orcblemf  it  is 
relatively  easy  t-o  reenter  the  existim  problem  structure  to 
chanoe  values  or  to  a'^d  or  remove  orotlem  dimensions  as 
reauireo  (Barclay  et  al./  \^11 ,     o ,       vii). 

It  should  bf*  e"nDhasi7ed  that  in  no  sense  does  decision 
analysis  renlace  decision  makers  or  the  role  of  human 
judoment  in  decision  rnakina.  Intuitive^  implicit^  or 
judamental  decision  makina  is»  after  all^  the  mainstay  of 
the  experienced  decision  maker  and  for  good  reason. 
Intuitive  un-aided  methods  have  tyoically  served  the 
decision  r^^'Kfir  well?  and  he/she  can  reflect  on  some  personal 
history  of  productive  reliance  on  their  developing  intuition 
(Brinxers^  1^7?,  p,  1*^1).  There  is  considerable  evidence 
to  show  that  unaided  decision  rrakina  is  reasonablef 
effective^  anrj  reliable  ("^eterson  and  Beach/  19b7f  pp. 
?q-a(b).  However,  what  rjecision  analysis  does,  is  nrovide  an 
orderly  ani  more  easily  understood  structure  that  helos  to 
aaqregate  the  wisdom  of  experts  on  the  many  tooics  that  may 
be  needed  to  make  a  decision/  and  then  suooort  the  skilled 
decision  maker  by  orovidinq  him  with  sound  techniques  to 
suoolement  and  ensure  the  internal  consistency  of  his 
judgment . 

Complex   lecision   problems   are   often   qifficult     to 
resolve.    This  occurs  for  a  number  of  reasons.   Oct  ions  are 
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not  always  clee'^ly  aefine'i.  ^ny  results  which  may  be 
derive^  frcr.  the  selection  of  a  Particular  option  Tiay  be 
hiahlv  uncertain.  Also#  it  is  otte'^  aifticult  to  :Jeternine 
relative  creferences  tor  the  c^ossinle  aecision  outco^ies. 
'/'he'^  orobleTs  such  as  these  do  occ^  r ,  the  decision  maker 
norrrally  takes  steps  to  structure  the  orohlen^  and  reduce  it 
to  a  Tiore  explicit  forrn.  This  is  exactly  what  decision 
analysis  does. 

decision  analysis  Ouilos  uoon  four  Pasic  ele^^ents  whicn 
aro  inherent  in  any  'decision  Droblem  (I.^arc1ay  et  al.#  1977^ 
o.  I).  The  use  of  these  four  elen^ents  allows  for  a  s^iooth 
procedure  in  the  resolution  of  complex  decisions.  The  four 
elements  ere : 

t.  A  set  of  initial  courses  of  action.  You  must 
have  more  tnan  one  alternative  or  there  is  no 
decision  to  be  made.  All  possible  alternatives 
snould  be  considere(i  without  reqard  to 
plausibility  at  this  ooint. 

•?.  The  Dossiole  conseaijences  of  each  initial  act. 
These  must  be  considered.  What  are  the 
important  thinas  that  can  happen  that  will  make 
one  act  more  valuable  or  worth  more  than  another 
act?  Relevant  seauences  of  subsequent  events 
an^  -follow -UP  acts  must  be  identified  for  each 
initial  act, 

3.   How    attractive    or    unattractive    is    each 
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co'^se'aijence  o^   each   act?  How   desirable   or 

undesirahlp  is   cop   outcc^e  comoared  to  others 

wnich  ,Tn-j.-!ht:  r^sijlr   froT   ^'^e  sai^e   or   a'^otner 
decision. 

'4.   How  likely  is  it   tl^a^   a  oarticular   act   will 

res 'J  It    in   each   o^   the  conseauences.    Tnis 

oroOabilitv  or  uncertainty  can  be  measureo  as   a 

crocaoility  fro'T^  0  to  1  or  in  the  form  of  odds. 

These  four  eleTients^  as  oescribed^  orovide  a  way  to 
orqanize/  ouantify/  and  trace  the  logical  i-noli  cations  of 
the  decision.  Ihe  primary  oojective  is  to  provirle  a  model 
of  at  least  oart  (all  would  be  best)  of  the  cjecision.  The 
use  of  the  woro  Todel  i  "^  this  case  "leans  to  represent  the 
decision  in  a  Quantifiable  form. 

It  is  a  central  oreceot  of  oecision  analysis  that  all 
relevant  consi"iera»:ions  in  a  oecision  can  oe  represented 
fully  in  a  decisiori  diai":raT!  (Barclay  et  al.f  1977»  o.  2). 
Tnis  decision  liagra'^  will  show  everythinq  a  decision  maker 
feels  is  relevant  to  the  problem  in  question.  A  decision 
diagram  consists  essentially  of  a  network  of  branches 
corresponding  to  Possible  sequences  of  acts  and  events^ 
fanninq  cut  from^  an  origin  at  the  left  to  a  time  horizon  at 
the  riqht.  Acts  are  available  choices.  Events  are  possiole 
occurrences  which  are  partly  or  completely  outside  the 
decision  maker's  control^  though  the  chance  of  one  of  them 
hapoeninq   may   be  influenced  by  acts  which  were  carried  out 
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earl  i  e  r . 

Tne  Tecision  •iianraT'  crgchic^lW  'listinquishes  acts 
frof^  events.  Act  for<s  -^r?  reoresenteci  by  sou  a  res  ana  event 
f  o  r  '<  s     are  represented  by  circles  as  shown  in  Fiqure  1. 

These  act  forks  and  event  forks  are  then  comhine:J  to 
form  a  decision  tree.  An  exatiole  of  a  tree  is  shown  in 
Fiqure  ? ,  T-iis  decision  situation  involves  a  cc^oany  which 
rnust  flecil'^  whothor  to  bid  on  two  orojectSf  ^^  anri  H,  The 
decision  on  croject  ^  must  be  maoe  nrior  to  the  aecision  on 
B. 

This  <,iecision  diaqrarr  depicts  all  the  ocssicle  acts  and 
pvents  and  shows  how  these  relate  to  each  other  in  the 
decision  situation.  v hile  the  representation  would  help  a 
decision  maker  to  see  at  a  nlance  his  alternatives  and 
inentify  those  thinos  t'^at  mioht  affect  any  choice  to  be 
rrade»  It  does  not  yet  answer  the  central  Question:  which 
choice  should  be  made?  That  question  can  not  be  answered 
without"  consi'ierim  tne  value  of  the  possible  outcomes  and 
the  likelihood  of  occurrence  of  the  events.  In  order  to 
show  how  this  is  incorporated  into  the  decision  models  let's 
return  to  the  oiidinq  exaTiple.  Let's  say  it  has  been 
calculated  it  will  cost  '5  10, 0  0  to  prepare  a  bid  for  either 
project.  If  the  bid  on  A  is  won,  a  qain  of  5  50,000  will  be 
realize f^.  Projec''  6  will  also  return  a  oain  of  fSOrOOO  if 
the  bid  is  won  hut  nue  to  overhead,  will  onlv  return  a  Qain 
of  B2^,0  00  if  bio  A  is  also  won,  Tne  decision  tree  now 
looks  like  Fi-^ure  3.   The  numbers  shown  at  the  ric^ht  side  of 
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FIGURE  1:   ACT  AND  EVENT  FORKS 
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FIGURE  2:   DECISION  TREE  STRUCTURE 
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FIGURE  3:    DECISION  TREE  WITH  VALUES  ADDED 
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the  diaore-"  are  the  oath  values.  These  values  are  the  sum 
of  the  costs  and  gains  a1onT  each  oath. 

^iC  ^  f  o  ccToletio  the  oiaa^sTi/  we  nee  a  to  oortray  tne 
cieqree  of  uncertainty  about  the  events-  we  cannot  control^ 
soecifically  whether  a  oiH  is  won  or  lost.  Based  uoon  orior 
exoerience^  it  is  judneo  t-hat  the  bid  on  oroject  A  is 
eduallv  likelv  to  be  won  or  lost  (orobability  ,S  win/  ,5 
lose).  If  A  is  won,  the  cron?bility  of  winning  the  bid  on  B 
is   reduced   to  ,  ^ .  '-'owever,  if  '^  is  lost,  the  chances  of 

winnim  ^  increase  to  .7,  and  if  ^  is  not  bid  on,  the 
chances  for  6  are  .o  to  win.  Adding  these  values  leaves  our 
diagram  looking  like  ^idure  '^  * 

This  diagram  should  now  be  a  virtually  ccTolete 
translation  or  mogel  of  the  rercection  of  the  decision  Taker 
of  the  decision  nroble'^.  All  that  is  left  now  is  to 
dpterrrine  which  is  the  best  solution.  The  Tiethod  to  do  this 
involves  tne  calculation  of  weighted  values,  often  called 
exoec^'ed  va'ues,  for  each  decision  ootion.  This  technigue 
is  the  si'^olest  and  s^'atistically  ^ost  straightforward 
method  that  can  oe  used.  Care  Tust  be  taken  however,  since 
frequently  there  are  decision  circumstances  wherein  an 
expected  value  solution  to  a  decision  oroblem  may  not  be  an 
ootimal  one.  In  thpse  cases,  alternative  means  of  treating 
value  ^re    required  and  will  be  discussed  later, 

Ceterminina  a  value  for  eac^  act  in  a  decision  is  done 
by  a  orocedure  called  folding  back  the  decision  diagram. 
Values  are  substituted  for  each  act  and  event  fork  beginning 
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at  the  riqhr-hanT  side  of  tHe  niagram.  At  eacH  event  fork/ 
the  exoected  value  of  that  event  is  calculatei  by 
multiolvim  th<?  value  of  each  nossible  outco^^e  by  the 
orobabilitv  of  the  occurrence  of  that  outcome.  At  eacn  act 
fork/  the  act  witn  the  hiohest  value  is  the  one  which  should 
aive  the  oest  decision.  The  crocess  of  substitution  is 
continued  until  the  initial  act  fork  is  reached.  Figure  5 
denicts  this  nroce'^urp.  Therefore  the  decision  rT.a'<er  would 
bid  on  "jroject  '*^/  then  if  he  /<<on  tne  hig/  not  bid  on  Project 
R.   If  he  lost  the  oi'i  on  A,  he  would  Did  on  project  B. 

Thus  far  in  tne  discussion  of  decision  analysis*  the 
values  of  the  gains  or  losses  were  assigned  in  terms  of  only 
one  measure/  mon^sy.  The  outcomes  were  then  comcared/  using 
the  exoected  valu^/  very  easily  since  it  is  easy  for  us  to 
relate  to  money  as  a  value.  However/  in  most  real  life 
situations/  esoeciallv  those  of  the  military/  gains  and 
losses  ar  pf  not  measured  in  money/  rather  in  terrain/ 
eguioment,  caoabilities/  and  even  lives  (i«'nlliams/  I'^l  6  f  d. 
?.2),  These  v^l'je  dimensions  are  oualitative  rather  than 
guantitative  and  are  very  difficult  to  measure.  To  further 
cloud  the  issue/  each  individual  has  his  own  oersonal 
subjective  values  for  each  of  these. 

In  addition/  verv  seldom  are  these  value  dirnensions  or 
attributes  civen  egual  weioht  in  a  decision.  For  examole/ 
consiger  the  selection  of  a  radio  set  from  several  comoeting 
models.  Cost  mav  oe  an  imcortant  consideration/  but  so  are 
weight/   range/   oortaoilitv/   ang   reliability.    Just   how 
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iTinortant  each  is  requires  aetemination,  /'hat  is  imoortant 
in  the  iTecision  is  which  of  these  attricutes  is  more 
i^nortant  than  tho  r,f'-;ers  and  how  is  this  difference  scaled. 
Then  now  oo  the  remainino  atrrioutes  fit  on  this  scale.  For 
instance/  are  cost  and  reliability  eaually  irnrortant?  If 
not  f  what  exactly  is  the  difference  and  how  does  the 
decision     rr^a^  e  r  make    trade-offs    between    the    two. 

f^pferminaticns  of  this  sort  must  {-e  maoe  for  all  attributes 
and  soT^ehow  our  mooel  must  ce  an)e  to  comoare  across  these 
many  att'^ibutes  and  aaarenat'=>  the  results  to  inoicate  the 
best  solution. 

Proble.Tis  of  this  nature  have  SDurretJ  the  cjevelopment  of 
multi-attrioute  utility  models  (^'lilliams^  197  8/  d.  2^). 
These  mcjels  nelc  to  crovide  a  relative  rankinq  for 
attributes  and  also  crovide  a  common  guide  for  aaoreqating 
the  measures  into  a  sinale  index  of  worth.  This  orocess 
involves  fhe  assionment  of  a  measure  of  utility  or  merit  to 
the  attributes. 

L'tility  can  be  described  as  a  subjective  measure  of 
"Mkinq"  (Parclav  et  al.,  \''^11 ,  o.  27).  It  is  a  personal 
value  reflecting  how  you  subjectively  value  something.  The 
acolication  of  utility  as  a  measure  for  the  value  of  an 
attribute  or  alternative  was  first  orooosed  by  Von  Neuman 
and  '^lorgenstern.  Tf^ev  suaoested  that  eacn  individual  has  a 
measurable  preference  among  various  choices  available.  The 
preference  they  calleo  utility  and  is  measured  in  arbitrary 
units   called   utiles   (Von  Meu'^an  and  '''oraenstern,  1'^'4  4^  p. 
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?7).  Utility  is  base'1  on  the  conceot  that  in  decision 
Tskina/  ^  r-3'*son  ■will  choose  that  alternative  y^hich 
m^Yirri^^s  nis  or  her  ''xoected  utility  . 

Multi-attrioijte  utility  tiodels  have  been  used 
extensively  in  the  systen^s  acauisition  role  (vvilliams/  1978# 
D.    2^).  The   oroceoures   encouraae   discussion  among  the 

decision  'r^a^  er  and  his  staf^,  N  o  longer  can  cost  be  a 
little  Tiore  iToortant  than  reliahility.  It  is  now^  oerhaos/ 
ten  utiles  rore  imocrtant.  Once  the  values  are  assigned  to 
tne  Tiodel*  an'i  a  rest  decision  decioe''i  uoon,  the  rrooel  now 
alloi/^s  you  to  vary  thp  relative  weiohts  to  see  which  ones 
have  the  Tiost  irnoact  on  the  proposed  solution.  This  is 
called  a  sensitivity  analysis  and  helos  decision  makers  when 
they  are  uncertain  about  the  accuracy  of  their  infor. -nation. 

Decision  analysis  aoolies  ^ayesian  techniaues  to  place 
a  value  on  inforration  that  reouces  uncertainty  (Keen  and 
Scott  ''Norton,  I'^Th,  c.  ^i^).  Bayesian  techniques  orovide 
for-ral  Tethcdolooies  for  analyzing  the  imolications  of  a 
decision  mat<'er's  suojective  judgn^ent  of  probabilities/ 
uodating  these  assp'ss  Tents  as  additional  information  is 
obtained.  This  aoditional  information  can  result  in  one  of 
two  events:  either  the  uncertainty  is  comoletelv  removed  or 
the  additional  information  allows  the  rtecisicn  maker  to 
revise  the  initial  Corior)  assessment  of  the  probabilities 
(Keen  and  Scott  '^^orton,  1^7^,  n.   ab). 

As  stated  earlier^  the  use  of  decision  analysis  does 
not  reolace  the  decision  ma^er  but  ados  a  new   dimension   to 
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his  decision  nnaKino  caoabilities.  It  is  also  assum^'i  the 
use  of  tnese  "lanual  technioups  could  slow  the  decision 
T^akinT  oroaress  consioerablyf  est^ecially  if  a  fear  of 
Tathof^atics  exist-s.  f^o^^l.lna'■plY/  what  has  t^een  develooed 
are  decision  aids  which  areatly  ease  the  coTiplexity 
i  n  V  o  1  V  e  cj , 

A  decision  aid  is  a  human-syste^  interface  desivqned  for 
the  soecific  ourrose  of  suDoort-inq  ani  enhancing  a  decision 
Taker  in  his  decision  -rai^ina  role  (Keen  ana  Scott  '^■orton, 
I'^T^f  n,  58).  It  is  a  ton!  for  use  by  the  decision  maker. 
Decision  aids  ^r^^  nor-nally  stereo  on  connuters  as  tho  use  of 
the  cor^outer  reduces  all  needs  for  calculations  by  the 
decision  ma<er  and  helos  to  SDeed  the  decision  process. 

The  general  availability  of  low  cost/  high  caoacity/ 
fast  information  orocessina  technolooy  has  enabled  man  to 
exten:)  a-^d  increase  his  intellectual  cacacity.  The  effect 
of  this  has  been  that  man  can  now  deal  more  effectively  with 
comolex  matters  on  his  own  comolpx  terms  as  aoainst  the  more 
simolistic  terms  of  pre-in'formation  technoloay  man 
(Brinkers/  1972/  p.  5).  The  focus  is  not  on  the  com outer 
itself/  but  on  the  technolooy  for  dealing  with  the 
information  that  is  available  and  the  scientific  methods 
which  are     in  develooment  and  use. 

The  use  of  decision  aios  normally  olaces  increased 
demands  uoon  human  creativity  and  judgment  rather  than 
relievino  the  decision  maker  of  t  ne  need  to  exercise  them  as 
mioht  be  exoected.   This  is  because  the  availability  and  use 
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of  oecisiof^  ^\  ^.3  which  se^'ve  as  an  extension  of  ti^e  decision 
maker's  intellect-  provide  new  an'i  interestina  ODoortunities 
for  usino  nis  inte11<=^ct.  Decision  ai^is  helo  the  decision 
T^aker  ov  orescrihina  a  straightforward  norrrative  crocedure 
for  oroanizinq  and  analv^ino  difficult  decision  oroolems 
involvina  bot^  unce'"*'aintv  about  the  outcome  of  future 
events  and  oernlexity  about  the  complex  value  tradeoffs 
inv/olved  in  the  choice  of  a  course  nf  action  (brinkerSf 
l<?7^,  o.   7)  . 
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TI.  II3F  PF  ^-F^I^THN  M^S  .MTHiri  ThF  C^  AO^Ma 

CoTimand,  control  en^  communications  (C3)  can  De  Jefined 
as  a  orocess  which  orovides  the  commander  (or  decision 
maker)  with  a  means  of  receiving  information,  making 
decisions  haseo  on  this  information,  and  then  imolementing 
^^^  ^^.^^^Pinn  the  decision  in  or-^er  to  achieve  his  or  her 
mission  C-oose,  l^-^O).  ^  C3  system  includes  the  data 
aatherino  (or  in^eHioenc-)  crocess,  t he  dec i s i on  mak i ng 
process,  and  the  oceratinq  forces  (or  weaoons)  reauired  to 
achieve  the  mission.  A  C3  svstem  would  look  somewhat  like 
the  figure  shown  oelo". 
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This  figure  sho'-^s  i-hat  data  is  qatneredf  oossioly 
thro  u  ah  th<^  use  of  e^rly  '/varnina  raoars>  ohotograohy,  or 
reoorts  fro  I-,  .-^it-hin,  ^n-i  sent  to  a  co"^-Tiancj  center.  Once 
wit"hin  the  co'^rr^ncj  center,  the  data  is  analvzeJ  and  a  de- 
cision is  made  reoardinn  the  use  of  weaoons  or  units.  These 
decisions  are  fhpo  transTitted  to  the  resoective  units  for 
imrle'Tientation.  Peduirements  for  data  or  inforrpation  could 
orioinate  a*"  the  command  center  or  it  could  he  snontane- 
ouslv  oenerated  as  in  thp  case  of  a  missile  att'aci<. 

(Jnfortunatelvf  the  majorit/  ("if  net  all)  of  the 
resources  soent  in  th»  irporovement  o  'f  these  systems  are 
exoendea  in  either  orovidinq  cetter  data  aatherina  (such  as 
better  ra^'ars)f  failsafe  communications/  or  biqaer  and 
better  weancns  systems.  These  are  all  important  aspects  and 
should  fie  i  T  Drove  d,  out  this  is  also  indicative  of  the  lack 
of  aoorec  iation  for  what  must  be  done  ^ith  t-he  data  when  it 
is  delivered  to  the  command  center.  Little  thouaht  is  given 
to  now  the  decision  maker  orocesses  the  the  data. in  order  to 
make  effective  decisions  and  forecasts. 

The  amount  of  data  available  today  to  decision  makers 
is  mindboodlinq.  PeaHsticallvr  if  methods  for  cognitive 
information  orocessina  are  not  improved  at  the  same  rate  as 
our  communications  can  provide  it^  much,  if  not  all  of  our 
sophisticated  communications  technoloav  will  go  under 
utilized.  /vhat  is  needed  is  more  research  aimed  at 
imcrovina  our  information  manaoement/  decision  making,  and 
forecasting  utilizino  tho  aoplications  available  through  the 
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coTDuter  ('inrjriole/  r^>^0,  n.   1), 

Decision  Tiakcina  lips  3t  the  he=irt  of  the  C3  crocess 
('Vpririole/  \''^^''jf  o.  1?).  The  entire  CS  system  focuses  on 
orov/i'iino  ^  Teans  to  transmit  all  reauirpo  data  to  the 
decision  maker  anH  then  a  mpans  to  transmit  the  aecision  to 
the  reauired  oarties.  Therefore^  C3  svstems  which  do  not 
contain  so^^e  sort  of  decision  making  suooort  should  be 
chall^meo  because  they  ere  inromolete. 

'"'est  Decision  makers  will  begin  with  a  need  for  some 
bac<arouno  inf'^rmation  on  the  oroblem  at  h^nd.  Has  there 
been  a  oroblem  1ik-e  tnis  oefore?  How  have  the  people 
involved  reacted  in  similar  situations?  Are  there  any 
oeoole  available  who  have  exoerience  in  this  area?  These 
are  all  e^^aTcles  of  the  tvoes  of  Questions  which  will  need 
tobeanswered. 

Nevtf  tho  decision  n^aker  will  want  all  of  the  latest 
facts  Surround  in  d  the  oroblem.  f'lho  is  availaole  to  move 
into  the  area?  *■^ow  lona  will  it  take?  .^iho  is  there 
already?  '.'hat  forces  does  the  oooosinc  side  have  available? 
v\hat  is  haooening  now?  These  ancj  many  more  are  the 
Questions  surround  1  no  the  current  situation. 

Nowf  the  decision  maker  will  want  to  know  what  the 
oDtions  are  and  what  the  adversaries  options  are.  In 
addition,  will  be  the  need  to  know  of  any  activities,  either 
in  orocess  or  oendinc/  whic^^  mav  nave  an  impact  on  the 
decision.  Mext  is  the  assessment  of  the  orobabilities  of 
tne   oossiole   outcomes   of   these  activities.   The  criteria 
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which  .pvill  be  USPH  in  analyzing  the  elternativ^^S  must  be 
listen  ar^'i  relative  ircortance  weights  for  these  deterTiined. 
Finally  the  rjecisicn  -na'x  e  r  evaluates  the  alternatives  and 
selects  a  course  of  action  to  be  followed. 

How  does  ti^e  decision  Taker  do  all  this/  esoecially 
when  the  i-rcac^s  of  a  wrona  decision  can  be  so  costly?  It 
certainly  can  not  be  'Jonp  alone.  The  decision  maimer  must 
relv  on  «-he  aid  of  i^'ey  staff  members  who  have  certain 
exoertises  in  soecific  areas.  But  even  with  comoetent 
staffs^  the  task  is  often  too  areat.  Comoijterized  decision 
aids  may  be  the  only  answer. 

There  are  several  reasons  why  comouterized  decision 
aids  would  oe  useful  in  the  C3  decision  ma<ina  crocess 
(Alien  et  al./  1976/  cc.   2-5  ).   These  reasons  are: 

1.  It  is  an  effective  tool  for  improving  the 
dialooue  amonq  staff  officers  who  are  workinq 
tooet'^er  on  a  oarricular  oroblem.  It  would  be 
heloful   in  eliminating  misunderst'andinas. 

2 .  The  orocess  of  using  a  decision  aid  gives  staff 
officers  ^he  ocportunity  to  include  what  they 
considers  important  in  the  analysis.  Then  they 
can  see  how  these  factors  actually  imoact  on  the 
solution, 

3.  It  is  very  easv  to  evaluate  several  courses  of 
action.  Variables  which  are  uncertain  can  be 
chanoed  to  allow  for  the  best  and  worst  cases  in 
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the  gnalvsis. 

ii.  Critical  are=^s  can  now  be  ■'ocui^ed  on.  ^  ]  ]  too 
often,  too  Tuc^  ti-ne  is  waste  ("i  in  discLJSsina  or 
arouino  lAihat  Tiav  seem  to  be  important  points, 
but  turn  out  to  be  rather  insignificant  in  the 
final  solution, 

5.  Tire  Tiav  no^u  hp  in  your  favor  ratner  than  your 
ooconent.  ""'ost  oeonle  do  not  react  well  when 
under  tifre  constraints.  Peocle  tend  to  over 
reac*"  to  unimoortant  eve'~'ts  ann  TiaWe  auick  and 
hasty  judT^ents.  Decision  aicts  enable  and  force 
the  rjpcision  Talker  to  follow  a  systematic 
routine.  Tnen  the  aid  nerforms  the  necessary 
calculations  in  a  real  time  manner.  This  oives 
additional  ti'^e  to  either  wait  for  additional 
information  or  actively  seek  for  it  if  the 
analysis  shoi»'S  a  neen. 

There  are  numerous  examples  of  C3  problems  where 
comPuteri7eT  decision  aids  would  be  useful  and  the  following 
are    some  examolest 

1.  Decision  aids  can  and  are  being  used  to  monitor 
earlv  warning  systems  for  any  indications  of 
impending  crisis.  I^  is  important  to  not  allow 
events  to  control  us,  but  rather  to  possibly 
have  us  shaop  the   events   (Andriole,   I'^hO ,       o. 
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30)  . 

?.  ^  recent  e  x  ^rvnl  ^  c^nc»rne")  t^e  analysis  of 
r^Tfjorf^nt  evacuation  oostures  that  a  cof^rnancier 
was  consiaerino  in  the  face  of  uncertainty  about 
a  develooinn  crisis  that  cou1J  have  made  it 
necessary  to  -evacuate  U.S.  nationals  from  a 
third  country  C '  '•  iHiannSf  1^78,  o.   ^7  )  . 


3.   The  nopTial  continoency   olanninq   process   that 

constantly  tai^es    olace   is   another   examcle 

f  ."-■  i  ]  1  1  a  m  s  ,  1^7  ^  ,       D  .    ^  ^  )  .    The   o  e  c  i  s  i  o  n   aid 

allocs  you  to  easily  olav  the  "what  if"  aame. 

^ ,  Traininrj  is  another  evar^ole.  By  allowinq  staff 
officers  the  chance  to  use  the  aids  based  on 
past  crises/  they  beco-ne  familiar  with  current 
plans  ana  procedures  and/  if  realistically  used/ 
become  use'^  to  oceratim  in  a  crisis  environment 
(All  en  et  a  1  . /  1 ^7o/  p.   S)  . 

5.  Decision  aids  may  be  used  for  establishing 
policy.  notions  can  be  evaluated  and  tested 
prior  to  deciding  on  firm  policy. 

6.  Tne  desian  of  new  C3  systems  would  be  another 
example.  /ihenever  new  systems  are  oesioned/ 
there  is  always  a  tradeoff  of  caoabilities 
because  of  insLjfficient  fundina.  These  aids 
would    helo    determine    the   safest   cuts   in 
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caoahilities     to    he    -na^e 
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in.  F  X  r  s  T  T  N  G  c  3  0  p:  c  I  s  1 0  N  a  i  o  s 

This  chaotpr  a escribes  several  C3  computer-oasea 
information/  iecision^  and  forecasting  systems  desioned  to 
HnK  the  human  user  with  the  information  so  critical  to 
efficient  information  manaaement. 

Tnere  oresently  exists  much  too  little  emohasis  uDon 
tne  content  an -J  esoociallv  the  form  of  the  information  and 
data  that  flow  thro u ah  fS  systems  (Andriole*  1"^80/  o,  3). 
The  duestions  of  ho'^  the  information  should  aooear  to  the 
user,  how  the  information  should  be  stored/  retrieved/  and 
manipulated/  ano  ho^  the  ever  increasino  amounts  of 
information  can  be  comprehended  by  the  user  have  not  been 
adeauately  dealt-  with  uo  to  this  point.  The  three 
comouter-oas'^d  information  systems  described  below  were 
developed  with  thes«»  auestio^s  in  mind.  Each  is  aimed  at 
helpinT  tho  C^  svston  users  deal  with  the  enormous  a^^^ount  of 
information  for  which  thev  are  responsible.  Tne  systems 
were  developed  with  the  human  user  in  mind  and  thus  exploit 
and  supplement  existinq  human  information  processing 
caoab  i  1  i  t  i  es , 

Soatial  Oata  Base  '"''anaaement  -  The  use  of  normal  data 
base  management  systems/  from  the  perspective  of  the 
occasional  user,  are  difficult  and  inefficient  (Andriole/ 
1980/  p.  3J,  Recently/  research  has  produced  a  prototype 
spatial   data   management   system   which   enables   users   to 
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hierarc^^ically  storer  rptrieve^  and  maninulate  data  through 
tne  use  of  soatiel  controls  (^noriole/  19 '^O,  p.  a).  These 
soatial  controls  allow  the  user  access  to  data  -without  the 
need  or  uS"  of  the  conventional  keyboard.  Instead/  the 
soatial  controls  oenmit  access  to  data  using  numerous 
navaoational  aidS/  such  as  colore  location  (in  an  electronic 
worksoace  orojected  on  a  large  screen  displav)»  touch,  and 
sound.  ScecificaMv/  t'^e  user  "nay  store  and  retrieve  the 
data  accord  in  o  to  /vhere  he  or  she  finris  it  Tost  easy  to 
access  . 

The  soatial  da^  a  nn.^naaeT.  ent  systeTi  was  develooed  in 
accordance  with  the  way  hu^ians  normally  store  and  retrieve 
information.  For  examolef  in  a  normal  office  environment, 
oeoole  store  information  in  familiar  claces  according  to 
freduencv  of  use,  imoortance,  shaoe,  size^  ana  so  forth. 
The  orototvne  soatial  data  base  manaoement  system  uses  these 
osychological  oredisoositions  by  allowing  the  user  to  create 
his  or  her  own  osvcholooical  wori^snace  ancj  oroject  it  onto  a 
laroe  ^isolav.  This  oisplay  then  is  interactive  and  caoable 
of  storing  all  tyoes  of  data  to  i nc 1 uae  numeric/ 
ohotooraphic,  and  even  audio.  Tt  is  also  easy  to  update. 
The  user  is  then  able  to  manipulate  and  retrieve  data  by 
movina  throuah  the  workspace  with  joysticks  or  touch 
sensitive  disolav  oenels.  Figure  7  shows  the  hierarchical 
data  oraanization  principle.  Figure  ^  shows  a  user 
positioned  in  *ront  of  a  soatial  data  base  management 
system.   Figure  ^  shows  the  joystick  and  touch  pane)  mounted 
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FIGURE  7 :   HIERARCHICAL  DATA  STRUCTURE 
(Andriole,  1980,  P.  5) 
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FIGURE  8:   THE  PROTOTYPE  SPATIAL  DATA 

MANAGEMENT  SYSTEM  (Andriole, 
1980,  P.  5) 


35 


o 


D 


en 


a 


a 


0 


FIGURE  9:    DATA  ACCESS  CONTROLS 

(Andriole,  1980,  P.  6) 
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in  the  arr  of  a  user's  chair.  Data  is  accesseri  by  touchino 
a  specific  noint  in  fhp  lA/ork-soace  or  the  touch-sensitive  Pad 
ao'j/or  hy  Tiovino  the  joystic'<  for'>/ar'i  or  oacl<waro  (to  ascend 
Of"  f^esceno  into  the  hierarchically  orcsanizea  aata  base). 

A^iaotive  C3  Information  Selection  -  In  resoonse  to  the 
ever*  increasinq  inforfnation  require  nients  olaced  upon 
intelliaence  officers  responsible  for  analyzing  and  routing 
infomation,  the  '^daotive  Infor-riation  Selector  CATS)  was 
developeo  (-'adni  et  al.,  l^y^,  d.  1-3).  Durinq  this 
developT'ent  orocesSf  it  ^as  determined  that  "new  C3 
techniques  are  re-.^uiren  to  control  inforrnation  flow  so  as  to 
best  match  svsten-  capability  with  human  characteristics  in 
the  Tan-comouter  interaction". 

The  ^IS  process  is  as  follows.  The  system  user  is 
calibrated  during  a  training  phase  to  determine  how  and  why 
some  messaqes  (pieces  of  information)  are  selected  for 
further  analysis  and  some  are  rejecten.  This  is  performed 
for  eacn  in 'J  i  virtual-  user  such  that  he  or  she  has  a 
personalized  infor-^ation  manageTient  sorting  ^  routino/  and 
Gueuina  syst'^m. 

i/^ihat  haooenSf  in  effect/  is  the  computer  monitors  the 
user  and  then  internalizes  how  the  user  selects  and  rejects 
information.  Once  fhe  training  program  is  accomplishedf  the 
computer  emulates  thp  user  and  automatically  selects* 
routes*  and  queues  the  information.  Thus  the  AIS  may  be 
viewed  as  an  information  management  assistant  capable  of 
relievino    its    superior   from   some   of   the   burdens   of 
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i  n  f  o  rti  ^  t  i  on    overloa'i. 

jhe       fu 'actions       a^^n       elements       nf       an    A  IS    are       presented 
araohically     in     Fici.ir'=>     10, 


Training  Channel 


At  Tribute  Levels 


ADAPTIVE 
UTILITY 
MODEL 


MESSAGE 
UNIVERSE 


Attribute  Weights 


MAE 


i 


INFORMATION 
SELECTOR 
MECHANISM 


Automatic  Channel 


Manual  Channel 


FIGURE  10:    ADAPTIVE  INFORMATION  SELECTION 
SYSTEM  CONCEPT  (Andriole,  1980, 
P.  8) 


The  AIS  has  been  develooed  for  use  where  C3  information 
is  already  in  an  electronic  form.  The  AIS  is  designed  to 
suDcort  areas  .^hich  are  already  comolicated  w  i  t  »^  too  nnuch 
incoming  in^orTation  for  the  human  user  to  handle.  The  AIS 
is  merely  a  computer  nroaram  /^hich  provides  helo  to  the  C3 
system  user. 

Ultra-Rapid  Readina  -  Even  if  systems  such   as    soatial 
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6s  t  a  base  mgnaoenent  ann  AIS  c^jn  auic<lv  orovi'le  access  to 
nepTe'i  intofnation,  t-n*^  user  -nust  still  roao  anrj  cofricrehena 
the  infoPTi^f-ion  fioforp  furthpr  ac^'ion  can  be  contemolater). 
SoDSeau^ntly/  r='searc^  has  oeen  perforrreri  to  imorove  the 
soeed  with  which  users  can  rean  and  comorehend  fAndriolef 
1980,  DO.   ^-11). 

A  technique  knoi«n  as  raoi.i  serial  visual  presentation 
(^SVP)  has  c'^'^n  oeveloneo  and  conouterized  for  use  in  C3 
svstei^s.  The  text  is  presented  one  Aord  at  a  time  on  a 
cathode  rav  tuhe  (C^T)  in  r  eo  ^  n  succession  as  Fioure  11 
suggest  s . 
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ULTRA   I 
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FIGURE  11: 


ULTRA-RAPID  READING 
(Andriole,  1980,  P.  10) 
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Research  has  in-iicoted  that  sinole  f^SVP  sentences  can 
be  rea'-J  ^n^i  accjr-it-oly  recalle-i  when  si^own  at  a  rate  as  hiah 
as  \  i  wons  OPT  secorif  which  is  twice  as  fast  as  oeople 
normany  r^a-i  f^ntter  et  al./  1 '^  7  ^ ,  o.    1). 

h?eadinQ  sneed  and  coTicrehension  is  a  -^unction  of  t^e 
time  it  tal^es  for  the  eves  to  transrrit  to  tHe  brain  and  the 
brain's  ability  to  orocess.  The  ultra-ran  id  reader  was 
oeveloned  to  shorten  th^  time  between  the  eyes  and  the 
brain.  It  does  this  ov  reoucinq  eye  rpovexent  to  a  bare 
'pinirru''.  The  eves  do  f^ot  need  to  ^love  bac'<  and  forth  or  ud 
ana  down,  as  is  norTally  done/  since  the  text  a c pears  on  the 
C^^T  in  the  saTip  location.  In  addition/  substituting  some 
oictures  or  symbols  for  woros  has  increased  comorehension. 
However/  as  wit^^  AIS/  the  text  must  oe  in  electronic  form  to 
be  read  u  1  t  r  a  -  r  a c  i  cj  1  y  . 

The  orecedina  three  research  desians  were  directed 
towaro  imnrovioT  the  information  flo«  to  the  decision  ma'<er  , 
The  ne*t  three  examples  are  computerized  necision  aids  which 
are  desianed  to  be  used  primarily  during  crisis  situations. 
Each  thermits  the  user  to  select  and  retrieve  history  on  past 
crisis  situations.  The  best  way  to  describe  each  is  by 
providinq  a  samole  of  the  types  of  Queries  which  are 
allowed.  The  first/  as  a escribed  in  Figure  12  f  enables  a 
user  to  retrieve  and  prescribe  crisis  actions  (or  responses) 
and  objectives.  The  second  aidr  as  shown  in  Fiqure  13/ 
enables  a  user  to  focus  on  the  manaoement  problems  which 
occurred   durinc   oast  crises.   The  third  aid  enaPles  a  user 
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to  examine  307  crises  involvino  the  U.S.   between   l^ata  and 
1^7fo,  33  Fiaijro  l^'-l  sho^s. 

These  ails  were  develooeti  in  direct  response  to 
reauests  fro-"  crisis  manaqers  for  so'-.e  tyne  of  on-line 
analytical  assistance  (Anciriole.  '^ay-June  1979,  d.  IS). 
Crisis  rranaqers  are  now  able  to  assemble,  sort,  and  analyze 
crisis  rranaae^ent  data  on  a  real  time  basis.  Preliminary 
nerformance  tests  of  these  aids  suooest  they  might  increase 
the  ran':!e  of  oction  neneration  and  evaluation  dunna  crisis 
hy  100%  and  reduce  oast  search  case  time  by  50%. 


THIS  SECTION  IS  DESIGNED  TO  ASSIST  DEPARTMENT  OF  DEFENSE  PERSONNEL  IN 
EVALUATING  PROPOSED  COURSES  OF  ACTION  AND  SETS  OF  U.S.  OBJECTIVES  BASED 
ON  DATA  FROM  101  CRISES  INVOLVING  THE  UNITED  STATES  BETWEEN  1956-1976. 

THE  USER  IS  GIVEN  THREE  LEVELS  OF  ANALYTIC  ASSISTANCE: 

1.  THE  CAPACITY  TO  SEARCH  FOR  HISTORICAL  CASES  WITH 
SETS  OF  USER-SPECIFIED  U.S.  ACTIONS  OR  OBJECTIVES. 

2.  THE  CAPACITY  TO  IDENTIFY,  ACROSS  ALL  CASES. 
THOSE  ACTIONS  THAT  HAVE  HISTORICALLY  BEEN 
MOST  STRONGLY  ASSOCIATED  WITH  EACH  OBJECTIVE 
SELECTED  BY  THE  PROGRAM-USER; 


3.       THE  CAPACITY  TO  IDENTIFY,  ACROSS  ALL  CASES. 

THOSE  ACTIONS  THAT  HAVE  HISTORICALLY  BEEN  MOST 
COMMONLY  ASSOCIATED  WITH  SETS  OF  U.S.  OBJECTIVES 
SELECTED  BY  THE  PROGRAM-USER. 

WOULD  YOU  UKE  TO  SEE  A  SUMMARY  OF  ANOTHER  SYSTEM  SECTION? 

PRESS  Y'  OR  'N'  AND   RETURN'. 
Y 

ENTER  T.  'II'.  OR  '111'. 
II 


FIGURE  12:    CRISIS  MANAGEMENT  DECISION  AID 
(Andriole,  1979,  P.  16) 
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THIS  SECTION  PERMITS  A  DETAILED  EXAMINATION  OF  MANAGEMENT  PROBLEMS 
ENCOUNTERED  IN  41  SELECTED  CASES  11956-1976). 

MAJOR  PROBLEM  CATEGORIES  ARE: 

1.  SYSTEM-RELATED  DELAYS  IN  DECISION-MAKING 

2.  SYSTEM/PROCEDURAL  CONSTRAINTS  ON  ACTIONS 

3.  LEGAL  ISSUES  INVOLVED 

4.  RESOURCES  INADEQUATE  FOR  DECISION-MAKING/ACTION 

5.  INTELLIGENCE  FAILURES  AT  DECISION-MAKING  LEVEL 

6.  EMOTIONAL/IDEOLOGICAL  ISSUES  INVOLVED  IN  DECISIONS 

7.  INTERPERSONAL  FACTORS  IN  DECISION-MAKING 

8.  PROLONGED  CRISIS  PROBLEMS 

9.  PROBLEMS  IN  SELECTING  ACTION  PERSONNEL 

10.  CONSTRAINTS  ON  OPERATIONS 

11.  PHYSIOLOGICAL  PROBLEMS  FOR  OPERATING  FORCES 

12.  INFORMATION  FAILURES  BY  OPERATING  FORCES 

13.  FAILURES  IN  TAKING  APPROPRIATE/TIMELY  ACTION 

14.  FORSTAT  PROBLEMS 

15.  PROBLEMS  IN  THE  OPERATING  ENVIRONMENT 

16.  GENERAL  PROBLEMS  IN  CRISIS  PLANNING 

17.  GENERAL  PROBLEMS  IN  CRISIS  HANDLING 

18.  GENERAL  PROBLEMS  IN  CRISIS  TIMING 

WOULD  YOU  UKE  TO  SEE  A  SUMMARY  OF  ANOTHER  SYSTEM  SECTION? 

PRESS  'Y'  OR  'N'  AND   RETURN'. 

Y 

ENTER  r.  Ml',  OR  'III'. 

ill 


FIGURE  13:    CRISIS  MANAGEMENT  PROBLEM  ANALYZER 
(Andriole,  1979,  P.  17) 
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THIS  SECTION  PROVIDES  ACCESS  TO  INFORMATION  ON  307  CRISES  (1946-1976). 

THE  DATA  MAY  BE  SEARCHED  FOR  CASES  MATCHING  THE  USER'S  SPECIRCATIONS. 

OR  A  FULL  DESCRIPTION  MAY  BE  PRINTED  FOR  ANY  SELECTED  CASE. 


CATEGORIES  OF  INFORMATION  CODED  FOR  EACH  CRISIS  ARE: 

YEAR  AND  BRIEF  DESCRIPTION 
LOCATION  OF  CRISIS 
'nature  OR  PRE-CRISIS  ACTIVITY 
DURATION  OF  PRE-CRISIS  PERIOD 
SCOPE  OF  CRISIS  (DOMESTIC  OR  INTERNATIONAL) 
NATURE  OF  CRISIS  (MILITARY. POLITICAL.BOTH) 
CRISIS  DURATION 
TIMING  OF  CRISIS  RESOLUTION 
CRISIS  OUTCOME 
ANTICIPATION  OF  CRISIS 
DEGREE  OF  THREAT  TO  U.S.  INTERESTS 
TIMING  OF  THREAT  DEVELOPMENT 
TIME  AVAILABLE  FOR  DECISION 

SIZES  OF  PARTICIPANTS/DEGREE  OF  U.S.  INTERESTS 
U.S.  RESPONSE/PARTICIPATION 
U.S.  OBJECTIVES 
NUCLEAR/NON-NUCLEAR  IMPLICATIONS 


WOULD  YOU  UKE  TO  SEE  A  SUMMARY  OF  ANOTHER  SYSTEM  SECTION? 

PRESS  'Y'  OR  'N'  AND  'RETURN'. 
N 


FIGURE     14 


CRISIS  DESCRIPTOR 
(Andriole,  1979,  P 
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Ourim  crisis  sifuat-ionSf  decision  rnai<.ers  strive  to 
react  swi^tlv/  deci"*?  '-/iselvf  and  coTimunicdte  accurately. 
Tnis  requires  c1os«=  coordination  with  their  staffs  and  the 
aCility  to  overcome  certain  odstacles  such  as  oressures  from 
time  constraints*  amhiouity  of  qoals*  and  the  monorolization 
of  time  with  information  collection.  Some  of  the  problems 
of  information  collection  f^ a\/ e  been  solved  or  at  least  aided 
by  the  LiSf^^  of  one  or  T^ore  o*  the  oreviously  discusseo  aids. 
*^jhat  the  "iiecision  m^kor  now  neeis  are  effective  ciecision 
strateQies  that  imoose  riqor  and  orovide  a  loqical* 
structural  fra'^ework  to  assist  them  in  tne  orocess  of 
choosina  an  ootimal  decision  alternative  in  the  face  of 
voluminous  and  often  inconclusive  evidence. 

Operations  an.,i  In^olHaence  (OPPiT)  is  a  decision  tool 
that  orovi^es  Just  such  a  framework  for  ^deliberation, 
reasonina,  and  analysis  (Arrey  et  al.»  I'-^T^,  d.  3).  OPINT 
is  an  on-line*  interactive*  real-time  model  which  aids 
decision  makers  bv  orescribing  a  straiqhtforwarc  normative 
orocedure  for  oroanirim  and  analyzing  difficult  decision 
problems.  These  problems  may  involve  both  uri certainty  about 
the  outcome  of  future  events  and  perplexity  about  the 
complex  value  trade-offs  involved  in  the  choice  of  a  course 
of  action. 

0^1 '^iT  is  a  decision-analytic  baseo*  computer  assisted 
decision  aid.  Its  orin^ary  objective  is  to  provide  decision 
makers  a  orocedural  framewor<f  or  decision  template*  that 
insures  thpir  ultimate  decision  choice  is  a  coherent  one.   A 
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coherent  cnoic*:^  is  one  which  is  consistent  with  their  own 
value  structures  an^i  beliefs  ^hout  the  relative  livelihoods 
o'^  future  evonrs  th^t  will  imoact  the  decision  outcon^e  (Arrey 
et  al  .  ,  1 ^7Q,  n.   3)  . 

The  funriaTiental  oroduct  of  OPirjT  is  a  coniouter-stored 
conceotual  reoresentation,  or  decision  models  of  the 
decision  orohle'^i  at  hand,  whereas  decision  analysis 
orovides  the  theoretical  dac^qround  and  procedural  guidance/ 
the  QPI'iT  decision  model  provides  the  soecific 
•nethodoloaical  tool  for  orocessina  information  and 
evalua*:ino  the  various  decision  alternatives  ooen  to  the 
decision  maimer.  An  in-derth  exclanation  of  how  OPINT  aids 
in  the  decision  process  is  discussed  in  Charter  Four, 

Evaluation  (t:V"L)»  another  on-line/  interactive/ 
real-time  Tociel/  incorporates  another  decision  strategy 
which  nrovides  the  derision  maker  with  a  normative  procedure 
for  analyzing  difficult  decision  problems.  It  is  especially 
desianed  to  evaluate  and  comnare  possible  options  usinq  a 
multi-attribute  utili«-y  analysis  technioue.  EVA  I.  is  best 
applied  to  procurement  (decisions  wherein  systems  are 
compared/  but  is  aporopriate  for  a  diverse  set  of  decision 
analytic  problems  (Decisions  and  DesianS/  Inc,/  1*^77/  pp, 
1-5), 

The  use  of  EV^L  is  broken  into  two  phases.  First/  the 
decision  maker  creates  the  structures  o^  a  hierarchical 
multi-attribute  evaluation  model  using  the  tVAL  Structure 
oroaram.    Then/  this  model  is  evaluated  and  processed  using 
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the  EVAL  nroaram. 

In  Qfier  to  utilize  t^^e  EVAL  Structure  orooram,  the 
oronleni  Tiust  be  ■neco'^onse^  into  a  hierarchical  structure 
which  reflects  th^  Iotic-^I  interrelationship  of  all  the 
factors  involved.  This  structure  is  then  entered  and  the 
model  is  create cK  Now  unaer  the  direction  of  the  EVAL 
nroara'T'f  the  criterion  used  for  evaluatino  the  alternatives 
are  entered.  Once  entered/  their  rpsoectiv**  weiuhts  of 
i-noortance  are  entered.  Mext/  the  relative  score  which  eacfy 
alternative  achieved  for  each  criteria  is  entered.  For 
examole/  consider  thre=  radio  sets  which  are  being  evaluated 
for  selection  before  nrocurement  (Al,  A2/  and  A3),  The 
criterion  of  evaluation  art^  cost/  ranae/  oortabilitv/  and 
reliability.  Each  of  these  criteria  are  weighted  30%/  2  0'/, , 
10%/  ann  ^0%  resoectivelv.  ^^ext/  the  three  radio  sets  are 
evaluated  accoraina  fo  each  criteria  using  an  aroitrary 
utility  scale.  Cost  would  show  scores  of  5  0  for  set  A],  aO 
for  set  A2,  and  <^0  for  set  A3.  This  means  A3  is  considered 
to  have  the  best  cost.  This  process  is  repeated  for  each 
criteria.  It  is  also  normal  for  the  criteria  to  have 
sub-criteria  which  would  also  reauire  the  same  procedure. 

Mext/  EVAL  gives  the  evaluation  scores  for  each  of  the 
alternative  systems  resulting  from  the  aggregations 
orescritjed  in  the  model  structure.  ''^low  the  decision  maker 
can  examine  the  effect  of  the  criterion  weights  throuah  a 
sensitivity  analysis.  This  will  quickly  show  him  which 
criteria  are  the  critical  ones  and  take  action  to  insure  the 
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data  surrounTinc!  rh^t  criteria  is  as  accurate  as  oossible. 
EV^L  ^oes  not  io  anvti-iino  tne  decision  maimer  nimself  could 
not  hev'*  jonp  hiTsel'^.  Nhat  it  does  do  t  ^o^eve  f ,  is  allow 
him  not  ♦■  o  be  oressed  bv  time  or  to  worrv  about 
ccPDutational  procedures.  He  can  olay  tne  "what  if"  qame 
and  feel  confident  about  the  decision  he  must  make. 

Another  decision  aid  has  been  neveloDed  to  assist 
B  arouDS  in  necidinq  how  to  deal  with  imoortant  issues  or 
'  oroblems.  Its  nuroose  is  the  imorovement  of  decision  makinq 
i  by  continuous  oarticicant  interaction  with  the  comouter 
durina  tne  aecision  makinq  process  CLeal  et  al.r  1978,  d. 
l-l).  '^embers  o^  the  a  roue  ar<s  allowed  to  input  their 
respective  estimates  of  the  occurrence  of  specific  events 
and  their  valu-^s  (weiqhts)  of  the  importance  ot  specific 
decision  outcomes.  The  decision  aid  then  compares  the  input 
and  informs  tne  arouo  members  of  any  disaqreements  between 
them.  but  most  imoortantly/  it  focuses  the  discussion  only 
on  the  disaqreements  w^ich  are  important  to  the  stated  qoa1. 
[ne  differences  which  are  not  imoortant  are  shown  to  be  just 
that.  Finally/  it  suoqests  to  the  qroup  precisely  how  their 
disaqreements  miaht  be  resolved.  This  enables  qrouPS  to 
move  much  more  quickly  toward  decisions.  As  a  by-productf 
profiles  are  developed  of  the  participants  which  snow  who 
has  expertise  in  soecial  areas  as  well  as  who  is  a 
risk-avoider  or  risk-taker  (Leal  et  al.,  1978/  po,  1-5  to 
1-10).   The  function  of  ♦■ne  aid  is  shown  in  Fiqure  IS. 
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VIA  INDICATOR  UGHTS 
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DATA  ON  GROUP 

CONRJCTS 
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EVENTS  AND  MANIPULATES 
DATA  AND  DISPLAYS 


FIGURE  15:    THE  GROUP  DECISION  AID  PROCESS 
(Andriole,  1979,  P.  17) 


The  decision  qrouo  is  corroosei  of  the  oarticioants  (de- 
cision Ta^ersK  an  i  n  t  e  rred  i  a  ^  o  r  /  ann  a  director. 
The  interTediator's  nriTi^r^rv  function  is  to  facil- 
itate co-^-^unication  betv«/een  the  arouo  and  the  comouter. 
This  is  done  bv  translatinn  sncken  reouests  into  comout- 
er  inputs.  These  inouts  could  be  lists  of  events,  alterna- 
tive actions,  requests  for  disolays,  or  modifications 
to  any  orevious  inputs.  The  director  helos  with  the  outout 
by  oresentina  and  exolainina  it  to  the  pa r t i c i oan t s . 
The  director  also  focuses  the  group's  activities  and  insures 
the  incuts  are  appropriate.  As  a  result,  virtually  no 
oarticicant   trainino  is  needea  to  operate  the  systern,  and  a 
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arcup  can  beain  wori<-  en  it-s  decision  Droblem  almost 
i.-n.-Tieoiatcly   after  it  is  convened. 

-nether  con-Dutef'izPT  aid/  the  tarly  .Naming  ana 
Monitorina  SvsteT-  (F/'ANiS)/  has  heen  O'^veloDeri  to  aid  the 
Indications  ano  v^iarnina  (li/l)  corm unity.  E»'iA^"S  orovides 
then^  with  an  interactive  coiiputer  based  system  of  oolitical/ 
military,  ana  domestic  indicators  for  daily  rrnnitorinq  of 
international  ma  intra  national  affairs  (Daly,  1^578,  o, 
^).  The  System  nas  the  cacability  to  store  auantitative 
international  oolitical  dat3  from  t'^ho  for  most  countries  of 
the  world  ana  can  therefore  oe  used  to  do  retrospect ive 
analysis  as  <vell  as  current,  daily  IJ^/'.  The  data  is 
aaqreqated  ny  month,  aua^ter,  and  year.  EwAMS  has  the 
capability  to  track  sinale  countries,  combinations  of 
country-oairs,  reqions  or  the  entire  world  twith  these 
quantitative  indicators,  or  specific  reoions  may  be 
specified  (Oaly,  1*^7  8,  o.  b).  At-  present,  there  is  some 
doubt  as  to  whetner  £ v*i A '-^3  will  ever  be  fielded  because  its 
ooeration  is  extremely  slo^'.  But  the  concept  is  a  qood  one 
and  further  research  may  prove  beneficial. 

The  E/jAMS  is  COT,  prised  of  the  following  components 
(Andriole,  19B0,  pp.   ?M-3a): 

1.  General  Scans  -  A  aeneral  scan  is  an  aqqreqation 
of  countries  by  some  criteria,  e.g.,  a  scan 
rjefined  by  .-jeoaraohic  region  would  aqgreaate 
countries  by  arouos  such  as  Latin  America  or 
Middle    Eastern.    A   general   scan   allows   an 
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analyst-  fo  look  at  several  countries  taken 
♦"OQether  rat^^er  tf^an  Inokina  separately  at  T^any 
country-nairs.  If  the  imi  cations  for  the  yrouc 
as  a  whole  suaaest  unusually  hioh  activity/ 
tension,  or  uncertainty,  the  analyst  can  droo  to 
the  country-ny-ccuntry  level  or  even  track  the 
recent  activity  of  a  sinale  country  to  determine 
tne  source  of  the  rtisturoance  or  the  reqional 
level.  General  scans  thus  ma^e  the  system  much 
more  efficient  and  less  time  consuminn  for  the 
analyst. 

2  m  Quantitativef  Political,  '"'ilitary,  hconomic,  and 
Domestic  Indicators  -  Quantitative  indicators 
for  crisis  warning  include  foreion,  domestic, 
and  international  colitical,  military,  and 
economic  factors.  "''He  indicators  are  both 
dynamic  ( e v en t s -b a seo )  and  static  (attribute 
based )  . 

3 .  ^'  u  1  t  i  '''  ethod  Forecastina  Capability  -  The 
unified  multi-method  forecastinq  capability 
requires  the  system  to  oenerate  different  kinds 
of  forecasts  or  warninqs  via  different  methods 
for  different  events  and  conditions..  Such 
options  have  been  designed  with  reference  to 
Objects,  ooals,  and  methods  of  forecasting. 
Experimentation  is  still  beino  oerformeo  in  this 
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A':/'Jition^1    research  is   currently   heinq  perforTierJ   to 

ciesian  an  ^id  ^hicb  will  *ocus   on   Soviet   crisis   behavior 

since   vorld  .'sar       II.  This  aid  will  include  the  following 
tasks : 

1.  DevelooTient  of  an  inventory  of  Soviet  crisis 
manaqe^ent  b-^havior,  l"^'4c>-1979, 

2.  Collection  o*  oata  on  the  characteristics  of 
these  crises  to  show  the  nature  of  Soviet 
military  crises. 

3.  A  Tore  intensive  analysis  and  codinq  of  key 
Soviet  crises  to  identify  the  crisis 
environTonts  that  mav  affect  the  occurrence  of 
crisis  manaaeTent  oroblef^s  encountered  by  the 
Soviet  Union/  and  variables  describing  crisis 
objectives/  actions/  and  results  for  the  Soviet 
Union. 

y.  Statistical  analys'='s  of  the  characteristics  of 
Soviet  crisis  manaqement  ooerations/  the 
environrrent  in  which  thev  have  taken  olace/  and 
the  crisis  man^^oeTien t  oroolenns  encountered  by 
the  Soviet  Union. 

5.  Comoarison  of  these  statistical  analyses  with 
earlier   analyses   of   the   sa^ne  factors  for  the 
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U.S.. 

All  o^  tf^ese  'lecision  aids  Hr3ve  been  ciavertised  as 
beino  vaHiahle  to  ■•"iprision  Tiakina.  None  h^ve  been 
exDerifTient^Hv  ev=)1ueted  with  kncwledaeable  subjects  in  a 
decision  T^ax-ing  environment.  A  baseline  of  such  evaluations 
is  imoortant  to  establish  in  order  to  orovide  a  Ljseful  tool 
and  better  yet/  to  orovide  for  much  needeo  technology 
transfer  and  real  »/orldf  real  ti-ne  feedoack  on  the  value 
(utility)  of  tne  tec^^nolo'^y. 

QPI'lT  xas  selectp^  as  an  accrooriate  candidate  for 
e  xoe  r  i  rnen  t  B  t  i  on  for  a  number  of  reasons.  (-irst^  it  is 
readily  availaole  as  a  oackaae  at  tne  ^'ava1  Postaraduate 
School.  Second/  all  of  the  eauipment  required  for  its 
oceration  already  exists  at  the  School.  Finally/  due  to 
both  time  and  monetary  constraints^  the  procurement  of  other 
aids  was  unfeasible. 
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I V  .  FU^'CTIO^^L  OFSCf^IPTIO'i  OF  OPI'-]T 

DPI  NT  is  a  oecision-analytic  based/  comnuter-assisted 
decision  strateav  (^mey  et  a]  .  ,  l'^'  7^ ,  d.  y).  It  was 
desianed  and  built  hv  Hecisinns  and  Oesions  Inc./  of  '^'cLean/ 
Viroinia/  under  contract  to  the  Oefense  Advanced  Research 
Projects  Aoencv  (Di^'PA).  It  i^as  'written  in  the  APL 
orooraTiTiina  lanouaoe  for  the  IC''^  SI  0  0  coTouter,  In 
addition,  0°P'jr  has  recently  Oeen  rewritten  in  FORTRA'"'vl  aP 
for-  use  on  POP  11  series  corrcuters. 

The  general  nuroose  of  OPI'JT  is  to  aid  decision  makers 
bv  providing  them  a  caoability  to  construct/  store/ 
retrieve/  exercise/  and  refine  decision-analytic  models  of 
comnlex  decision  problems  tnev  face.  The  OPINT  decision 
model/  which  is  interactive/  real-time/  and  on-line/  is  an 
ornanizinq  frampwork  for  information  orocessing.  Decision 
analysis  is  a  methodoloaical  tool  with  which  the  decision 
maker  defines  and  exercises  the  OPPiT  model  to  evaluate 
decision  alternatives  certain ina  to  the  problem  (Amey  et 
a1./  1^70,  OP.   5-?l). 

The  OPI'JT  system  is  designed  to  be  used  interactively  by 
decision  makers  who  are  relatively  unsophisticated  with 
resoect  to  computer  i-echnologv.  Accordingly/  the  design 
satisfies  two  human-factors  objectives:  OPPiT  is  a  menu- 
driven  system  an'i  is  desianed  to  be  foroivina  of  pro- 
cedural errors  by  the  us^r. 
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E^ci^   rtecision  mociel  created  bv  the  user  nav  be  aiven   a 

ijnioue  label*  althouqn  each  is  constructed  usino  the  same 
aeneric  for"nat.  The  r'Odel  forrrat  is  shown  qrachicallv  in 
F  i  q  u  r  e  lb. 

The   forn^at    consists    of    the    foMowinq  ele^^ents 

which,  when  comDletely  SDecified,  uniquely  deHne  an  QPINT 
decision  model: 

1.  The  decision  -  A  short  label  defining  the 
decision  pr oblen.  This  label  is  also  aoolied  to 
the  decision  model  and  is  used  to  store  and 
retrieve  the  iiodel. 

d.  Decision  ^l^-prnatives  -  A  list  of  decision 
alternatives  available  to  the  decision  maker. 
Eacn  alternative  is  aoorooriatelv  labeled. 

3.  An  iJncertain  Future  Event  -  A  key  uncertain 
event/  Ef  that  will  influence  the  eventual 
outco'^e  of  the  decision.  The  uncertain  event 
is  attached  to  each  of  the  decision  alternatives. 

y.  Event  Outcomes  -  A  list  of  the  discrete  event 
outcomes/  each  aocroo  r  i  a«"  e  1  y  labeled/  that 
together  define  the  universe  of  oossibilities 
reoardino  the  occurrence  of  the  future  event. 

5.  Event  Probabilities  -  A  vector  of  orobahilities 
that  are  associated  witn  the  event  outcomes/ 
which  reoresent  the  orobability  that  an  event 
will  occur.  A  D'robability  is  a  measure  of 
uncertainty.   It  is  a   number   between   0  and  1/ 
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FIGURE  16:    DECISION  MODEL  FORMAT 


55 


fo. 


inclusive*  that  reorespnts  the  extent  to  which 
an  iniivi'lual  believes  a  future  event  will  occur 
However,  in  this  soecification,  oroD abilities 
are  exore!^se'i  as  a  oercentaoe  of  certainty* 
e  .  q  .  »  as  a  0  %  vice  0  ,  '4 , 

Decision  Outcomes  -  The  elements  discLJSsed  thus 
far  Hofinp  oossible  decision  outcomes.  Each 
decision  outcome  is  a  oairod  combination  of  one 
decision  alternative  with  one  event  outcome. 


The  remaining  three  elements  of  the  model  forrrat  are 
used  to  soecifv  tho  re'\3f}\je  conseouences  associated  with 
the  decision  OL'tcomes.  The  consenuence  of  an  outcome  is 
exoressed  in  t'^rms  of  the  relative  regret  that  would  be 
exoerienced  bv  t*^e  decision  maker  should  the  outcome 
actually  occur.   These  three  elements  are: 

1.  Decision  ("utccme  Criteria  -  '^  list  of  criteria/ 
each  aooronria''ely  labelea*  by  whicn  the 
decision  rnaker  would  judge  the  relative  regret 
associated  with  the  decision  outcome. 

2 .  Criteria  /•j  eights  -  A  vector  of  weights 
associated  with  the  criteria,  which  represent 
the  relative  contribution  of  the  criterion. 
Criteria  weignts  are  expressed  numericaHy  as  a 
oercentaoe  of  the  whole/  e.a,/  as  6  0%. 

3.  Reoret  -  r?egret  is  a  measure  of  the  conseauence 
of    a    decision   outcome.    The   total   regret 
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assicinea  to  a  decision  outcof^^e  is  a  weighted 
linear  corrihin^jtion  of  the  individual  cr"iteria 
rearets.  For  eaci^  criterion  am  for  eacn 
ciecision  outcorre/  the  user  must  soecify  a  value 
of  rearet.  A  renret  is  a  number  between  0  and 
-1/  inclusive^  that  represents  the  relative 
dearee  of  dissatisfaction  that  the  decision 
Tai<er  associates  with  a  particular  decision 
Oi.Jtcom'a.  Zero  represents  no  rearet/  -1 
represents  maxiTum  regret.  How ever/  in  this 
specification  reorets  are  always  expressed  as  a 
percentaae  of  the  maximum;  e.g./  as  -3  0  vice 
-0.3.   t^efer  to  Fiaure  17. 

This  completes  the  model  format.  The  decision  model  is 
completely  and  uniquely  specified  i«hen  the  elem,  ents 
described  above  are  definea  by  the  user.  The  input 
specifications  oescribinq  the  model  can  be  processed  to 
oro^uce  the  followinq  results! 

1.  Corr^binea  Value  Regret  ^'^atrix  -  £•  sinqle  matrix 
that  displays  the  total  or  comhineo  rearet 
associated  with  each  of  the  decision  outcomes. 
For  each  outcome/  the  combined  regret  is 
obtained  by  weiqhtina  and  aading  the  component 
reorets  contributed  by  each  criteria.  A  combined 
value  rearet  matrix  is  snown  in  Fioure  18. 

2 .  Expected   Value   ^-'atrix   -   A  sinqle  matrix  that 
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FIGURE  17:    REGRET  MATRICES 

(Amey  et  al .  ,  1979  ,P .9) 
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COMBINED  VALUE 

E^  (U\BEL)       E2  (LABEL)  E3  (LABEL) 
Dj  (LABEL)             0                  -17  -31 

D2  (LABEL)  -9  -9  -17 

D3  (LABEL)  -2^  -22  -22 

Di|  (LABEL)  -30  -26  -25 


FIGURE  18:    A  COMBINED  VALUE  REGRET  MATRIX 
CAmey  et  al.,  1979,  P,  10) 
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^. 


Hisolavs  the  weigHted  exoecteH  rearet^ 
associated  'wi^h  each  decision  outcome.  The 
exoectei  value  matrix  takes  into  account  the 
relative  likelihoocjs  of  the  event  outcomes. 
Expected  Value  Vector  -  A  vector  that  aisolays 
the  weiohted  exnecte-l  rearet/  associated  with 
each  o^     the  decision  alternatives. 


The  exoecteo  value  matrix  and  the  excecteo  value 
vector   (total)   are   disolayed  toaether,  as  shown  in  Fiqure 

There  are  two  sensitivity  analyses  that  are  useful  to 
the  user.  Both  are  based  on  the  exoecteo  value  vector  and 
are  described  as  follows: 

1.  Threshold  'Matrix  -  A  iiatrix  that  di  Sclavs  the 
eleven  exoected  value  vectors  Generated  by 
either  one  of  the  following  operations^  at 
t^^e  user's  ortion: 

a.  Varying  the  crcbability  o'^  a  nesignateg 
event  outcome^  from  0  to  100%/  in  steos  of 
10,  The  other  event  outcomes*  maintain 
their  oroportional  relationships  with  each 
other. 

b.  Varyino  the  weiaht  of  a  designated  criterion 
from  0  to  lOnv, ,  in  steos  of  10.  The  weights 
of  the  other  criteria  maintain  their 
oroportional  relationship  with  each  other  in 
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EXPECTED  VALUE 


E^  (LABEL)       E2  (U^.BEL)       E3  (U\BEL)     TOTAL 
0  0  -i|  -4 

D2  (LABEL)  -5  0  -2  -7 


D-^  (LABEL) 


D3  (LABEL)  -12 

Di^  (LABEL)  -15 


■1 
■1 


-3 

-4 


-15 
-20 


FIGURE  19:    AN  EXPECTED  VALUE  MATRIX  AND 

AN  EXPECTED  VALUE  VECTOR  (Amey 
et  al .  ,  1979  ,  P.  12) 
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the  saT^e   nanner   as  do       the   oroo  abilities 

'lescrif)<=ri  in  fhp  nrecedina  p.iraarapi^. 

In  roth  o'f  t!^o  above  cases,  the  least  reoret  disDlaved 
in  the  excecteo  value  nayoff  vector  is  identified  by 
an  asterisk.  'Jomallv,  the  decision  alternative  that  leads 
to  the  decision  outco'Tie  having  the  least  rearet  *«ill  change 
as  the  designateo  event  orooabiHty  or  criterion  weight 
is  increfrented  from  0  to  10  0%.  The  coints  of  change 
are  referred  to  as  threshold  c">oints  and  ^re  noted  on  the  nna- 
trix.   Fiaijre  -^O  shows  an  examole  threshold  rratrix. 


EXPECTED  VALUE  WHEN 
PROBABILITY  OF  E^  (LABEL)   IS: 

0    10    20    30    40    50    60    70    80    90  100 

Dj  (LABEL)     -57  -52  -45  -40  -34  -29  -23*-17*-ll*  -5*  0 

D2  (LABEL)     -51  -47  -42  -38  -34  -30  -23*-21  -17  -13  -9 

D3  (LABEL)     -27  -27  -27*-26*-26*-25*-25  -25  -24  -24  -24 

Di|  (LABEL)     -26*-25*-27*-27  -27  -28  -28  -28  -29  -29  -30 


FIGURE  20:   A  THRESHOLD  MATRIX  CAmey  et  al . , 
1979,  P.  15) 
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?.  ^'' an u ally  c*^-inae  event  Drohaoilities  -  The  user 
mgv  Tenprate  a  test  e^oecteo  value  vector  t  •^  a  t 
is  hasorj  on  an  ari-i*-rarilv  assioned  vector  of 
event  c  rohab  i  H  t  i  es .  The  user  fray  soecify 
several  iifferent  crobability  vectors  and  note 
t^e  resultant  exoectea  value  vectors.  An 
exaTole  of  the  disolav  is  shown  in  Figure  21. 


KOREAN  INTENT 

NO  ACTION   HARASS   SHOOT  DOWN 

CURRENT  LIKELIHOOD:       60  30  10 

ENTER  REVISED  VALUES:  30  60  10 

NEW  VALUES:  30  60  10 

IF  THESE  VALUES  ARE  CORRECT  TYPE  GO:    (GO) 

EXPECTED  VALUE/REGRET 

DO  NOT  FLY  -38 

MODIFIED  ROUTE  -26 

ARMED  ESCORT  -10 

HIGH  PERFORMANCE  A/C      -10 

NORMAL  MISSION  -29 


FIGURE  21:   MANUALLY  CHANGED  EVENT  PROBABILITIES 
(Amey  et  al.,  1979,  P.  16) 


OPINT   is   a'='siqned   to   oerform   the   basic    functions 
descrioed  below: 

1.  Maintain  a  library  of  OPINT  -r^odels  -  Store 
various  decision  models/  filed  oy  their 
associate 'J  labels. 
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Loa'-l  9n  existinq  nPI^jf  model  -  Disolay  the 
1 -ah els  o^  these  mo n els  store (i  in  the  Tiodel 
lior^rv,  and  oerniit:  the  user  to  retf^ieve  any 
'^esirei  -noiel.  The  loa-ied  Tiodel  is  referred  to 
as  the  current  model. 

Disolav  the  results  of  the  current  model  - 
Permit  the  user  to  examine  the  structure  and 
Cont*^nt  of  the  current  model  by  disolayirq: 

a.  event  o robab i 1 i t i es ^ 

b.  criteria  weiohts/ 
c  .   reore  t  s  f 

d.  combined  value  matrix* 

e.  exoected  value  matrix,  and 

f.  expected  value  vector. 

Wevise  the  current  model  -  Permit  the  user  to 
make  chames  to  the  structure  ana  content  of  the 
current  model.   The  user  may  revise: 

a.  event  orobabilitieSf 

b.  criteria  and  criteria  weights* 

c.  decision  alternatives* 
d  .   r eore t  s  *  and 

e.   combiner)  value  regret  matrix. 
Save  the  current  model  -  Permit  the  user  to   add 
the  current  model  to  the  model  library. 
Perform  sensitivity  analyses  -  Permit   the   user 
to   test  the  sensitivity  of  the  current  moaei  by 
(Hetermininn   thresholds   or   manually    changing 
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probabilities. 
7.  Create  c^  new  OPPiT  tiqcipI  -  PprTiit  the  user  to 
create  a  new  '^oriel,  which  then  becofrips  the 
current  mocjel.  Tne  user  creates  a  model  by 
sceci^vinf3  the  elements  requiren  for  the  model 
as  explained  earlier.  The  user  may  also  use  a 
reference  aamble  to  validate  the  assianed  values 
of  rearet. 
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V.  THF  ^F3IG^J  OF  THE  EXPFPIMFijT 

The  conduct  o'^  exoeriTipnts  u'^ina  cof^n^and  and  control 
decision  aids»  such  as  GPIMT,  present  srecial  nroblems  (Cain 
and   Poh,   1*^78/  o.  ft).    Historically/  the  evaluation  of 

comTand  and  control  svstef^s  has  heen  focused  on  hardware 
TeasureTipnts,  I.  ittlp  attention  was  naid  to  software/ 
esoecially  decision  aids,  ^s  a  result/  few  well  defined 
Quantitative  TeasureTients  ^xist  for  measurina  the  quality  of 
C5  decision  aids  (Sinaiko/  X'^ll ,  oo.  5-6),  Secono/  there 
are  no  "correct"  answers  to  a  decision  oroblem  (Daniels/ 
1*^77,  o,  Ih).  Rgfher,  an  aid  such  as  OPIfiT  is  a  coqnitive 
tool  to  assist  thp  decision  maker  in  revealina  and 
forTiuMzina  the  ele'^pnts  of  the  decision  proble'^  as  the 
decision  ■ng<er  oerc^ives  them.  Lastly/  during  the 
develocment  of  these  decision  aidS/  there  Has  been 
relatively  little  syneraism  between  developers  and  r>otentia1 
users  (Cain  and  Poh/  1978/  o,  6),  Users  are  often  not 
familiar  with  the  new  technology  and  are  unsure  exactly  what 
to  exoect  in  the  way  of  performance.  Developers  are  unsure 
exactly  what  the  notential  users  want.  This  occurs  because 
thev  aren't  sure  who  the  users  will  be  and  causes  a  very 
general  aiq  to  be  developed  -  on**  which  will  try  to  suit 
everyone's  n^^e-is. 

Any  exceriment  v>;hich  involves  the  use  of  decision  aids 
must   be   desinnf>d   carefully.    The   experiment   should   be 
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desianea  in  orrier  to  assess  both  the  capability  of  the 
technoloqv  and  i  t  <;  overall  utility  as  it  relates  to  total 
systPT  oerfoT'^ance  (;^anielSf  197  7,  d,  1'4).  There  are  four 
levels  o"^  exceriTents  varyina  in  foPTaliST  from  very  loose 
"free  olay"  with  only  subjective  judq-nent  outout  to  "highly 
structured"  with  cofriolete  soocification  of  conduct  of  trials 
and  o'J^'out  consistinc:  of  carefully  measureo  systeii 
attributes  i*^  arr    et  al.f  1^7^/  o.  id)  .  These   levels   froTi 

least  structure'^  to  rpost  structured  are '. 

1.  Validation  exoeriT^ents.  These  exoerifents  woula 
consist  of  the  deouaoing  of  hardware  or 
software.  The  feasibility  of  the  systen  to  work 
accordina  to  fixed  soecifications  is  determined. 
An  examcle  would  oe  the  debuoaing  of  the  OPINT 
software  orior  to  placing  it  on  line  for 
operational  testing  or  use, 

2.  Demonstration  experiments.  These  experiments 
would  be  somewhat  more  structured  than  the 
valiootion  experiment/  in  that  a  scenario  would 
be  followed.  The  terii  demonstration  refers  to 
the  tyoe  of  output  from  the  experiment.  The 
outout  would  be  mostl-y  personal  impressions  in 
the  minds  of  the  users  of  the  system  under 
demonstration. 

3.  Assessment  experiments.  These  are  experiments 
where  triads  are  conducted  over  a  wide  range   of 
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conlitions  with  little  control  over  sources  of 
error.  The  ao^l  is  to  obtain  an  idea  of  how 
'«ell  the  systerr  cerforms.  The  output  is 
S'.jbjective  opinions  of  the  exoerirrenters  and 
subjects.  Thpir  ooinions  of  how  worthwhile  the 
svsteTi  is  and  any  ootential  uses  are  recorded  as 
exne  r  i -nen  t  a  1  data. 

y.  Fvaluation  exoeri-nents.  These  are  the  most 
rioorous  tyoe  of  experiments,  Exoerinnental 
conditions  rjre  carefully  controlled.  ^^  number 
of  replications  are  oerformeo  followeci  by  a 
formal  analysis  of  numeric  measurement  data. 

The  assessment  -experiment  was  chosen  as  the  means  for 
determining  the  utility  of  OPINT.  No  prior  experiments 
concernina  the  utility  of  OPI'MT  have  been  performed  and 
therefore  no  baseline  currently  exists.  The  results  of  this 
experiment  should  fill  that  aao.  Subjective  impressions  and 
judgments  of  C^  users  a  r  <=>  what  is  needer:)  to  help  (Jetermine 
the  usefulness  of  OPRiT  and  that  is  what  this  assessment 
will  do . 

The  major  objective  of  this  experiment  was  to  assess 
the  utility  of  OPIMT  as  a  computerized  aecision  aid  in  C3 
applications.  Secondary  objectives  which  helped  in  this 
ooal  included  tne  followina: 

I.  To  assess  tne  extent  to  which  OPI^!T  facilitates 
thoro'joh  an.j  timely  decision  makina. 
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2.  To  invesi-iqate  the  user-QPI'jr  interface  and  to 
suaaest  re'^edi-al  -^ea^^ures  or  alternate  oesiqns 
as  'joorooriate. 

'h .  To  survey  user  ooinion  on  ti^e  operation  tjnd 
Dotential  usef<j1ness  of  QPpJT  in  C3  decisions. 

The  priTiarv  objective  of  OPI\'T  is  to  assist  decision 
makers  in  the  struct  urino  and  analysis  of  decision  nror^lems 
(Amey  et  al.r  \'^1^ ,  c.  lA)  .  One  could  imply  from  tnis  that 
a  better  decision  or  croiuct  is  delivered  t^^rouah  the  use  of 
OPI'IT.  The  question  vhich  is  now  immediate  iS/  "What  is 
better?"  Hetter  can  not  be  scenario  dependent  or  olayer 
deoendent.  It  must  be  qeneral  in  nature.  OPIMT's 
oerformance  may  be  be  measured  in  terms  of  time  (the  time 
from  oercention  of  oroblem  to  decision).  A  faster  decision 
is  often  times  needed  and  therefore  the  capability  to  do  so 
may  be  termed  " ao od  to  have",  But  it  is  only  "good  to  have" 
if  the  Quality  of  the  decision  remains  at  least  as  qood  as 
without  the  aid.  In  order  to  assess  this  feature^  a  method 
for  measuring  the  quality  of  the  decision  is  needed.  As 
stated  earlier,  the  "correctness"  of  a  decision  is  extremely 
difficult  to  measure  if  at  all.  In  this  exoeriment  no 
attemot  was  made  to  define  or  measure  this  attribute. 

'A' hat  remained  was  the  subjective  judqment  of  the 
subjects  involved  in  tne  experiment.  The  siibjects  were 
students  from  the  C3  curriculum  at  the  Naval  Postgraduate 
School.   C5  students  vere   used   since   the   assessment   was 
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perforTe'i  of  iDPIf^iT's  ahiHtv  to  help  with  C3  tyoe  DroDlems. 
These  stuaents  wero  aH  oDerational  officers  with  a  larqe 
arrount  of  excerience  in  onePrStional  prohlens  in  the  field. 
They  reoresenten  the  'irmvf  Air  Forces  Navv^  an,:l  ^^arines. 
They  were  in  their  last  (sixth)  Quarter  of  studies  and  had 
been  exoosed  to  nurrerous  aecision  models  and  aids.  The 
subjects  were  arouoed  into  oairs  to  fornn  a  decision  team. 
All  the  suoiects  had  been  trained  in  the  theory  and 
oceration  of  0*^T>)T  ana  had  received  at  least  three  hours  of 
instruction  incluoina  one  hour  of  hands-on  time. 

Ti«o  tyoes  of  decision  situations  were  used  -  time 
critical  ana  time  not  critical.  It  was  understood  that  the 
ranne  of  ti^^e  oossibilities  is  infinite  and  there  was  no  way 
to  assess  each.  Thereforer  the  two  extre^nes  th.at  could  be 
encountered  were  usen. 

Two  scenarios  were  followed  which  reouired  the  subjects* 
actinq  as  decision  maimers/  to  recommend  a  soecific  course 
of  action.  The  two  scenarios  reoresented  varying  comolexi- 
ties  which  *ace  a  C3  decision  maker.  They  were  carefully 
designed  to  avoid  the  alleqafion  that  they  were  either  OPINT 
or  subject  dependent.  The  first  scenario^  a  Cuoan 
blockade  oroblem,  was  rather  complex.  It  required  an 
in-deoth  assessment  of  intelligence  estimates  in  order  to 
predict  the  occurrence  of  an  uncertain  event.  In  ad- 
dition, the  evaluation  of  numerous  courses  of  action  was 
requireci.  The  second  scenario  was  an  air  reconnaissance 
oroblem.    This   one  was   somewhat   easier  as  the  kev  uncer- 


70 


tain  event  hai  ^Ire^dy  been  assesses  and  the  decision  maker 
was  only  required  to  evaluate  the  courses  o^  action  which 
were  availaole.  •-•  cooy  o^  each  scenario  is  included  in  Ad- 
D  e  n  d  i  X  ^  . 

The  subjects  clayed  one  scenario  using  OPR'T  and  then 
olayed  the  second  in  a  manual  mode  without  the  use  of  OPINT, 
The  schedLjle  was  varied  so  some  subjects  plaveci  the  dif- 
ficult Cuban  croblem  first  and  son-e  the  easier  reconnais- 
sance oroblem.  In  addition,  the  use  of  '■IPPJT  was  variea 
so  it  was  useti  alternately  first  or  second.  Fiaure  22  oor- 
trays  a  tvcical  schedule.  '^lotice  the  aaded  dimension  of 
the  time  critical  and  time  not  critical  factors. 

It  was  recognized  the  evaluation  measures  used  in  the 
exreriment  would  not  be  Quantitatively  measurable. 
Therefore  care  was  tai^en  to  carefully  select  both  measures 
of  effectiveness  ("'•^OF)  and  measures  of  nerformance  C^OP) 
which  suDoorted  the  aoals  of  the  exo^riment. 

A  measure  of  effectiveness  is  defined  as  those 
evaluation  -measures  which  indicate  the  contribution  of  QPINT 
to  the  overall  decision  orocess  fACCAT  Qoerational 
Evaluation  Task,  1^7  8,  d.  b).  •'^ieasures  of  oerformance  are 
defined  as  those  evaluation  measures  which  indicate  a  level 
of  technical  oerformance  relative  to  the  internal 
functioninc  of  OPIMT  itself.  3oth  are  aoclicable  to 
decision  aia  experiments.  The  '-OE  '  s  and  ''OP '  s  were  desianed 
so  as  to: 

1.   Pleasure    the    contribution   of   OPIiMT   to   the 
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aecision  Process^  not  ^he  decision  outcome. 

2.  Be  objective  so  '■^ev  Jo  not  introouce  nias. 

3.  Pe  si-nple/  to  ac'iieve  econo^^y  ana  ease  of  use. 

4.  Permit  detopTiination  of  which  functions  within 
the  decision  orocess  are  aided  and  under  what 
condi  t  i  ons . 

5.  Yield  results  in  the  exreri-nent  whicn  are 
extend iole  to  real  world  ooerational 
envi  ronmentS. 

6.  Be  operationally  olausible  and  intuitively 
unaerstandahle  bv  ootential  users  (ACCAT 
Ooerational  Evaluation  TasK,  1978,  no.   6-^). 

The  followinq  is  a  list  of  the  V'OF's  which  were    chosen 
for  use  in  the  e  x  oe  r  i  Tipn  t  J 


1.  Time  elaosed  in  arrivina  at  the  decision.  This 
includes  the  time  from  oerceotion  of  a  oroblem 
until  a  decision  is  reached.  In  addition/  the 
time  elaosed  in  the  actual  use  of  the  aid  will 
be  measured. 

2 ,  Icientification/elimination  of  non-essential 
information. 

"5.  Objectivity  in  determining  relationshios  among 
factors  contributing   to   the   selection   of   an 
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alternative  course  of  action. 

^.   Fisk   oerceivea   to   ne   associate  c:i    with    the 
decision. 

5.   Ease  of  use  of  the  decision  aid. 

b.   Fase  of  modification  of  data  within  the  nnodel, 

7,  Perceived  advantaoes/disadvanta:ies   qained   from 
the  use  of  th*=  aecision  aio. 

8.  Ease  of  recovery  from  user  errors. 

Data  «as  qatherea  on  each  of  the  above^  throuah  the  use 
of  Questionnaires.  The  exceotion  was  the  first  '^OE.  For 
this  oner  timps  w»='re  recorded  while  the  sut^jects  were  using 
OPI^lT  to  caoture  the  amount  of  time  required  to  build  the 
model/  time  to  evaluate  the  data/  and  total  time  to 
decision.  These  tines  were  then  compare ci  with  times  to 
evaluate  data  and  total  time  for  the  manual  operation. 
Three  questionnairf^s  were  used  to  record  the  ooinions  of  the 
subjects.  A  questionnaire  was  comoleted  after  the  olav  of 
each  scenario  and  one  comoletec^  after  both.  Samole 
Questionnaires  are     included  as  Appendix  C. 

Ourino  the  actual  olav  of  the  scenarios^  a 
consideration  was  how  to  satisfactorily  place  the  subjects 
into  the  mental  ro1<=>  required.  This  included  the  ability  of 
the  subjects  to  adeouately  assess  orobabilities  of  future 
events  and  the  rearets  reauired  in  the  construction   of   the 
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model.  This  researc'i  design  subsumed  this  consideration  by 
locatincj  orobabi1i'"y  estimates  within  the  scenarios  and 
oefinino  t^o  ooals  and  objectives  to  be  reached.  Subjects 
were  encouraqed  to  nrovide  their  own  regret  assessments  if 
they  had  exoerience  in  si^^ilar  orot)lems.  If  the  subjects 
had  no  previous  exoeriencer  an  individual  was  available  to 
act  as  a  senior  official.  The  senior  official's  role  was  to 
orovide  reoret  dat-a»  if  needed^  and  to  answer  questions 
concernir:!  uncertainties  or  am  biauities. 
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V I  .  THE  RESULTS  (IF  THF  EXPhRI^hMT 

Tnree  siqnifiC'^nt  results  from  the  exoeriment  were 
revealed.  First/  nearlv  all  (85  "-i)  subjects  stated  they 
liked  the  caoabilities  which  OPI'>]T  Qa\Je  them  anrj  would  use 
OPI^'T  in  the  future  if  it  was  available  to  them.  Seconds 
althouah  tney  liked  the  caoabilities  o^  OPINT,  all  (100%) 
stated  they  did  riot  like  it  in  its  oresent  form^ 
particularly  in  the  area  of  user-interface.  Finally/  a 
Droarammina  bua  (erf'or)  was  discovered  in  the  use  of  the 
sensitivitv  analysis.  The  sensitivity  analysis  permitted 
the  comoarison  of  only  three  courses  of  action/  show i no  the 
expected  regrets  for  the  others  as  zeros. 

It  is  imoortant  to  Keec  each  of  these  results  in  mind 
as  the  specific  results  are  read .  Wany  negative  comments 
were  conditioned  oy  the  statement  that  if  user-interface  was 
better/  or  if  the  sensitivity  worked  correctly/  the  comments 
would  be  cosit-ive.  The  cases  where  this  occurs  are 
appropriately  noted. 

The  sensitivity  analysis  prooramminq  "buq"  was  found 
during  the  play  of  the  Air  Reconnaissance  Scenario.  This 
scenario  had  five  possible  courses  of  action  which  needed 
evaluation.  OPI^T  handled  the  expected  value  and  comoined 
value  calculations  perfectly.  But  whenever  a  sensitivity 
analysis  was  asked  for ,  OPI^iT  would  not  compute  the  expected 
rearet  for   all   courses   of   action.    Additional   research 
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reveale':^  that  in  the  case  of  a  sensitivity  analysis  where 
the  orohaniHty  of  the  event  was  varie'-l/  OPIuT  would  only 
comoute  threp  courses  of  action,  '.'ihen  the  variation  of  the 
criterion  vveiqhts  «as  as<e'-i  for,  DPI  NT  would  corroute  four 
courses  of  action.  Any  reTiaiTiino  courses  of  action  received 
rearet  values  of  zero.  This  did  not  oerrfit  a  comolete 
evaluation  of  alternatives  and  had  an  adverse  effect  on  the 
feelinqs  of  thp  subjects  toward  the  decision  aid.  Although 
OPINT  had  only  o<»en  installed  on  the  POP  11/70  for  one  rnonth 
prior  to  the  exoerirrent,  orocjramminq  bugs  such  as  these 
should  have  been  d'^t'^cted  and  corrected  orior  to  presenting 
the  aid  as  operationally  ready. 

All  but  four  subjects  said  they  felt  the  scenarios  were 
realistic.  These  four  felt  that  priorities  established  in 
the  scenario  were  not  what  they  ^elt  they  shoula  be  and  that 
orior  decisions  Tade  did  not  see^*  realistic  from  their 
viewpoint.  Fhp  use  of  the  realistic  scenarios  coupled  with 
the  ability  of  the  suDJects  to  inout  their  own  probabilities 
and  regrets  facilitated  a  decision  making  environment  that 
was  as  realistic  as  possible. 

The  eight  "^^OF's  stated  in  Chapter  Five  were  evaluated 
through  the  use  of  questionnaires.  A  summary  of  the  results 
for  each  is  snown  below: 

1.  Time  elapsed  in  arriving  at  the  decision.  Times 
i^epe  recorded  in  order  to  capture  the  time 
subjects  were  spending  on  the  scenarios,  both 
with   and   without   the   utilization   of   OPI.'^JT. 


77 


Those  tiiT>es  were  recorded  not  to  rjetermine  if  a 
fast'^r  aecision  was  oeino  maae ,  hut  to  provide 
an  indication  of  ho^  lonq  it  took  to  build  the 
OPI^jr  no  del  anri  evaluate  it.  In  all  caseSf  more 
ti^e  was  soent  while  using  OPINT  than  in  the 
manual  state.  The  time  reouired  to  build  the 
CjPTMT  Todel  averaged  ^5  minutes  for  the  Cuban 
scenario  and  2^  minutes  for  the  Air 
Peconnaissance  scenario.  Tne  times  were  tightly 
arouoed  as  is  shown  bv  standard  oeviations  of  '^ 
and  ^  minutes  resoectivelv.  The  time  to 
evaluate  THE  0^1  NT  model  averaged  ?A  minutes  and 
M  minutes.  Aaain  si^all  stancjard  deviations  of 
3  and  1  minute  show  tiqntly  arouoed  data.  These 
times  are  verv  near  to  what  was  exoected  and 
Show  a  aeneral  ability  of  the  subjects  to  build 
and  manipulate  the  model. 

^.  Identification/elimination  of  non-essential 
information.  The  goal  here  was  to  Determine  if 
the  decision  process  was  aided  by  identifying 
non-essential  information.  6^%  of  the  users  of 
OPITjT  were  able  to  identify  some  non-essential 
information  while  only  5l/i  were  able  to  in  the 
manual  mode.  The  items  of  information 
identified  variecj  according  to  subject-specified 
probabilities  and  regrets.  An  adaitional  ^4%  of 
the   OPI\'T   users   stated   problems    with    the 
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sensifivitv  analysis  creventen  them  frofn 
oossiolv  i:-ien«"ifyinQ  any  non-essential 
information. 

5.  Objectivity  in  rieterminina  relations^ios  among 
factors  contrihutino  to  the  selection  of  a 
course  of  action.  The  aoal  of  this  MQE  was  to 
deternine  if  the  subjects  were  aided  in  the 
decision  orocess  ny  aeterni nation  of  which 
factors  were  the  ^ost  critical.  b'i%  felt  UPINT 
definitely  aided  in  the  decision  orocess.  These 
subjects  definitely  liked  the  way  OPI^^T  oromoted 
detailed  and  non-subjective  analysis  of  what  was 
aoina  to  haonen  and  whv.  The  rest  found  it 
confusinq  due  to  uncertainty  of  what  the 
aisolays  were  actually  tellinq  fhem.  63%  also 
stated  they  were  able  to  assess  the  criticality 
of  the  con<-ributino  factors  when  using  OPIfiT. 
The  evaluator's  observation  here  was  that  in  the 
manual  mod^f  one  factor  was  determined  to  be 
critical  and  then  the  rest  iqnored.  This  did 
not  occur  when  OPirjT  was  used. 

U,  RisK  oerceivert  to  he  associated  with  the 
decision.  This  '^OE  was  used  to  measure  the 
.jeoree  of  uncertainty  or  certainty  in  the  final 
decision.  8  5  7-  of  the  subjects  in  the  manual 
mode  said  they  made  the  best   decision   oossible 
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wHile  onlv  56°'.  in  the  OPINT  mode  f^lt  that 
certain.  Two  tyo?s  o^  comments  were  macje. 
Firs'"/  the  sensitivity  oroblem  was  again 
me^tioneo  as  a  hindrance.  Secona*  several 
subjects  felt  uneasy  about  what  the  numbers  in 
the  disclays  actually  meant.  For  example/  in 
th<^  exoected  valuer  combined  value/  and 
sensitivity  disolaysr  only  numbers  are  shown. 
There  is  no  explanation  of  what  ♦■hey  mean  or 
which  numbers  are  best. 

Ease  of  use  of  the  decision  aid.  This  ^'*C\t  was 
used  to  heln  determine  what  tyoe  of  user-OPIMT 
interface  existed,  (Jnly  31%  stated  that  (J  PINT 
nresenteo  disolays  in  a  manner  which  was  easily 
understood.  Sensitivity  and  combined  value 
disolays  received  the  most  comments  as  being  the 
most  difficult  to  understand.  Also/  there  were 
several  comments  concerning  the  inputs  reauired. 
f-^any  were  not  sure  what  was  actually  being  asked 
for  and  when  to  hit  the  carriage  return  and  when 
not  to.  Almost  all  ^5%  stated  the  disolays  were 
qenerated  guicicly  enough.  No  problems  were 
observed  here/  even  riuring  times  of  Peak  usage. 
Additional  comments  concerned  the  use  of  menus. 
More  efficient  movement  throuah  the  menu  levels 
was  wanted. 
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6,  Ease  of  modification  of  data  within  the  model. 
The  no-^1  here  vias  to  determine  how  easv  it  was 
to  correc*"  ♦■he  many  errors  that  untrained 
tyoists  will  m^ike.  QS^i  stated  it  was  easy  to 
maice  corrections  in  the  model.  The  only 
neoative  comments  concerned  the  use  of  the  menus 
for  this  ournose.  There  existed  too  much 
retracino  throuoh  menu  selections  to  make 
corrections  in  the  same  area  or  level  of  the 
mode  1  . 

7.  Ease  of  recovery  froni  user  errors.  Tnis  '^OE 
measured  how  well  the  CJPIi'^jT  oroaram  recovered 
from  incut  errors.  Here  the  goal  was  to 
netermine  if  normal  tynina  or  careless  errors 
caused  major  orohlems.  ^PX  of  the  subjects  who 
made  errors  felt  recovery  from  these  errors  was 
Door.  Comments  centered  around  some  simple 
mistatfos  whicfi  caused  major  orotilems.  During 
initial  lisolaySf  a  selection  of  a  choice  is 
made  by  movina  the  cursor  to  the  desired  choice 
and  then  hittinc  the  'X'  key  for  execution, 
however  in  six  cases^  subjects  pressed  the  'X' 
key  followed  by  a  carriaae  return.  This  caused 
the  orooram  to  halt  and  the  subjects  were 
reauired  to  begin  again.  Several  subjects  felt 
that  nebulous  innut  directions  caused  several 
errors   which   coula   not  be  changed  until  after 
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the  entire  moiiel  was  coToleted. 

Perceived  advantenes/disadvaotaaes  aained  froTi 
the  use  c'^  OPI'^-jT,  The  evclicit  consideration  of 
all  factors  through  the  ororrotion  of  a  aood 
thought  Drocess  was  the  comment  most  often 
stated.  Several  stated  they  liked  beina  forced 
into  a  logical  thought  orocess.  Alsor  the 
subjects  felt  that  OPIfv'T  orovided  them  with 
gocu  mentation  of  their  c^ecision  and  woulcj  make 
it  easier  to  oresent  the  outcome  ana  justify  the 
conclusion.  The  disadvantaces  listeo  for  using 
DPl^T  as  it  currently  exists  centered  around  the 
u  se  r  -  i  n  t  e  r  f  ac  e  oroblem.  There  exist'=»g  a  general 
lack  of  comfort  due  to  the  limited  aescriotions 
and  assistance  from  the  aia. 

An  adgitional  Question  concerning  training  was 
asked  of  each  subj**ct.  They  were  asked  to 
comment  on  the  amount  of  trainino  they  felt  was 
reguired  for  OPIMT  usage.  The  overwhelming 
resDonse  was  about  two  hours  of  structured 
instruction  on  the  theory  and  use  of  the  aidr 
followea  by  one  to  two  hours  of  hands-on 
demonstrations.  This  amount  of  instruction  was 
exactly  what  each  subject  received  crior  to  the 
exoeriment. 
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V  [  I  .  Cn-;CLiiSinr:S  /\^ID  PEC^MM^^M)  A  T  T  UTiS 

It  was  oeterTined  that  OP  INT  is  beneficial  to  the 
decision  rnai<er  in  thp  'jecision  makino  orocess.  The  majority 
of  the  subjects  comments  were  either  positive  in  relation  to 
(3°I'iT  or  »ou]  ri  havo  been  with  either  no  proqram  errors  or 
bp'tter  use'^-in'-erfpce.  OPI^:T's  greatest  advantaae  lies  in 
two  areas.  First,  if  ororrotes  (and  cossihly  even  forces) 
the  user  into  a  souno  thouaht  orocess.  In  the  heat  of 
crisis  or  even  just  day-to-dav  decisions/  it  is  often  too 
easy  to  forget  the  orincioles  of  Tood  decision  making  and 
sHc  to  scmetnina  less.  The  use  of  OFIi^iT  could  help  to  keeo 
tnat  frof^  hanoenina.  Second,  OI-^IijT  provides  its  own 
doc  <-J  "mentation.  This  documentation  is  then  available  to 
ore  sent  decisions  and  show  ^hy  a  Particular  (decision  was 
made.  It  is  also  easv  to  focus  on  the  critical  areas  and 
not  waste  time  on    the  non-important  ones. 

There  are  still  some  major  flaws  which  need  to  be 
worked  out  of  the  system.  The  first  and  foremost  is  the 
sensitivity  analysis.  Program.  buos  such  as  these  can  not 
exist  in  an  aid  if  it  is  to  be  used  properly.  Second,  is 
the  user-interface  problem.  The  aid  must  be  designed  for 
the  casual  user,  one  who  at  one  time  may  have  been 
proficient,  out  throuoh  lack  of  use  has  forgotten  the  key 
reauirements.  Additional  comments  concerning  wnat  is  needed 
or  what  is  beina  displayed  are    a  must. 
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Tho  movement  o -^  the  user  fhrouah  the  aia  via  the  menus 
also  nee^s  attention.  Users  shoula  be  niven  the  choice  of 
returnina  to  several  levels  rather  than  only  uo  one  or  r:lown 
one . 

Top  fine!  anO  by  no  means  the  least  imoortant  is  the 
oroblem  ot  crooram  halts.  Clothing  arieves  a  user  more  than 
havina  to  start  over/  esneci^Hv  after  a  qreat  deal  of  work 
has  alrearty  ooen  Hone.  The  svstem  currently  disolays 
numeric  incurs  for  verific=^tion.  fnere  is  no  reason  why 
a'ohaoetic  or  menu  choices  could  not  also  be  done. 

This  author  has  S'=»veral  recommendations  for  both 
imorovino  the  decision  aid  anri  for  future  experiments  with 
it.  '^s  stated  earlier,  the  aid  needs  to  be  designed  for  the 
octional  user.  This  user's  knowledce  of  the  aid  could  ranqe 
from  comolete  to  nothinn  deoendinq  on  amount  of  usaqe.  The 
aid  should  De  desioned  *or  the  two  extremes  and  even  for  the 
user  in  the  mi'5dl'='.  It  ne^^ds  to  be  both  user  friendly"  and 
"error  tolerant".  One  technique  could  be  optional 
verbosity.  For  each  disolav/  three  sets  of  comments  could 
be  oreoare'J,  The  decision  maker  then  selects  which  set  is 

desired  deoenoinq  on  need.  The  knowledgeable  LiSer  would 
receive  only  short  phrases  which  could  be  keyed  on  to 
generate  in ruts  or  disolav  results.  The  same  tvoe  of 
caoabilitv  wou'd  exist  ^or  the  other  tyoes  of  users.  This 
way  the  user  receives  only  what  is  needed.  Another 
variation  of  this  is  to  allow  each  user  to  specify  exactly 
what   comrnents  arf>       needed   to   insure   correct   usage   and 
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intercr^itflt  ion, 

Tho  adiition  of  so-^e  soi^t  of  "HELP"  corn-,  and  would  also 
he  useful.  If  a  lecision  va'^er  nets  stuci<  and  needs  soi^e 
orofTDtina  or  internreta''ion/  a  sinncle  recuest  woula  fill 
that  need.  User  confidence  could  imorove  areatly  with  this 
siTioleadoition, 

Once  DPI  NT  is  redesiqned  to  meet  the  needs  of  the 
usersf  as  stated  in  tho  results^  additional  exDeriments 
should  oe  conducted  fo  Teas  u  re  ad-iitional  caoabilities  of 
the  a \ n .  This  e^oeriment  dio  not  consiaer  whether  a  better 
decision  was  heinf]  made.  This  is  not  an  easy  problem  and 
further  researci^  ano  exoenimentation  is  called  for, 

Another  area  for  exoerimentation  has  to  do  with  the 
are-^  of  rearet  values.  QPp^lT  currently  is  desianecl  to 
handle  r^afets  rather  than  gains.  Some  subjects  casually 
stated  they  i^iaht  have  oreferreri  to  use  gains  rather  than 
rearets.  Additional  experiments  to  oetermine  if  the  use  of 
Gains  vice  regrets  has  an  impact  on  user  efficiency  is 
definitely  calleo  for. 

Finallvf  thouoht  Should  he  niven  to  orovidinq  a  hard 
cooy  ootion  to  users  who  have  only  a  soft  copy  (CRT) 
capability.  Further  exoerimentation  could  be  done  to 
determine  if  potential  users  want  or  even  need  a  hard  cooy 
caoabi 1  i  ty. 

The  underlying  premise  of  these  results  is  tnat  most  of 
the  trouble  encountered  in  netting  computers  used  in 
decision  maKino  comes  from  too  much   emohasis   on   bits   and 
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bvtes  ann  Tiini-r^al  attention  on  the  user.  The  use  of 
decision  suncort  systems  such  as  I'PI'iT,  o-ff^r  gn  oonort  unity 
th^t  is  both  trivia)  ana  itmense.  It  is  iTrrense  in  that 
they  can  e^iopri  the  ccnp uter  in  decision  rrakina  activities 
where  larqe  oavo^fs  could  be  orovided  and  trivial  in  that 
they  represent  no  major  advance  in  technology.  Ruildinq 
innovative  svste'r'S  will  be  a  difficult  and  ris^y  venture  for 
some  tirne  to  co'T^e.  The  future  of  decision  suooort  systems 
anri  ooerational  decision  aiis  deoends  on  some  adjustments 
that  both  trie  develocers  and  users  must  maice.  6ut  these 
adjus'^ments  ^r  °  smaM  ones;  and  i^re  mainly  attitudinal. 
Little  nevv  kno^leaae  is  required.  But  the  oooortunities  are 
all  substantial/  a^.i  t^e  oersonal  "^nd  organizational  rewards 
h  i  oh  , 
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APPEMQIX  A  -  SAMPLE  OPrm  SESSION 


%  o  o  i  n  t 

1)  Ann  Arbor 
(?)  Any  harclc  OP  y 
3)  AO.M  3  A 
Which  terrrinal  tyoe  are  you  usina?  2 

Please  select  one  of  the  followinq  ootions. 
OPTIO^! 

1  Disclav  results 

2  Edit  Ti  o  >■:'  e  1 

3  Sensitivity 
^  Loa-i  To.:Jel 

5  Create  or  aJd  to  model 

6  Save  Tiode  1 

7  ^Jew     inforrration 

8  End  session 
Enter  desired  oQtion:5 

Is  this  a  new  TiodelTCy  or  n):y 

Please  enter  a  title  for  this  n^odel:  [air  reconnaissance   ] 

A'hich  Dortion  of  the  model  woula  you  like  to  create? 
OPTION 

1  Generate  influence  iiaaram 

?  Generate  value/rearet  model 

3  Aid  Bavsian  indicators 

^  Return  to  main  menu 

Enter  desired  ootionM 

v'jhat  is  the  m^in  event  of  interest?  [korean  intent   ] 
korean  intent 


1-  [ 


} 


Please  enter  the  outcome  lacels  for  the  event  you  have 
called  korean  intent 

Enter  outcome  label:  [no  action  1 

Enter  outcome  labelt  [harassmpntl 

Enter  outcome  label:  [shoot  sown] 

Enter  outcome  label:  f  J 
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Please  enter  the  oroh^oilty  of  each  outco-^e  of  the 
event  vou  have  laoele-i  korean  intent 

no  action  harass^entshcot  down 
-bO  30  lu  

iMcr-Tigl  i  ze^:!  va  1  ues :  oO.O       5<">.0       10.0 

Tf  these  values  arp  correct  tyoe  go:   [aol 

/•jhich  Dortion  of  the  model  would  you  li<e  to  create? 
OPTION 

1  Generate  influence  diagram 

2  Generate  value/reoret  model 
5     Advd  Bavsian  indicators 

^  Peturn  to  main  t, enu 

Enter  desired  opt-ion:,? 


fou  will  be  asked  to  orovide  the  criterion  to  he  used 
in  your  value  juoriement,  the  actions  you  ^ay  ta^e^  ano 
the  associated  value/reoret  matrices  for  each  criterion 
Please  oress  return  to  cont-inue. 

Enter  the  names  ^or  each  of  your  criterion: 


Cri  terion: 
C  r  i  terion: 
Cri  terion: 
Cri  terion: 
Cri  terion: 


fai  rcraft-ai  rcrew 
(loss  of  informafion 
[national  influence 
tree  on  oroaram 

[ 


'^low  enter  the  labels  for  each  action. 


Action 
fiction 
Action 
Act  1  on 
Act  ion 
Action 


[  do  nn t   fly  ] 

[modified  route  1 

[amed  escort  1 

[hiqh  cerf  a/c  1 

[normal  mission  1 

[  ] 


You  will  now  be  asked  to  orovide  the  values  for 
each  of  these  value  matrices. 
Please  oress  return  to  continue. 

Please  enter  regret/value  matrix  for 
ai  rcraft-ai  rcrew 

no  action  h a r a ssmen t shod t  down 
do  not  fly  0  0  0--   

do  not  f 1 y  0.0         0.0         0.0 

If  these  values  are  correct  tvoe  ao:  [gol 
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morfiHe^")  route     0  -2  0     -ol^ --------  —  ---__- 

modifieorout-e  O.n      -2  0.0      -6  0.0 

T^  these  values  are  correct  ^voe  oo:  [qol 

ar^ied  escort       0  -15  -55 

ar^edescort  0.0      -15.0      -35.0 

It  these  values  are  correct  tyoe  qo:  [qo) 

hiqh  oerf  a/c      0  -5  -15 

hiah  perf  a/c  0.0       -5.0      -15.0 

If  these  values  are  correct  tvoe  qo:  f. qo] 

normal  Tiission      0  -<^5  -10  0--- 

norTialfnission  0,0       -25.0      -UHJ.O 

If  these  values  are  correct  tvoe  qo:  fool 


P  1  e  a  s  <=  ent^r  rea^e  t  /  va  1  ue  "natrix  for 
current  information 

no  action  harassfT'entshoot  aown 

do  not  fly  -100  -100  -100 

do  not  fly  -100.0      -100.0     -lOu.O 

If  these  values  are    correct  tyoe  go:  [qol 

■nodifieTi  route  -50  -50  -50 

rn  o  d  i  f  i  e  d  r  o  u  t  e  -30.0      -50.0      -  3  u  .  0 

If  these  values  are    correct  type  go:  Tool 

armed  escort  -5  -5  -5 

armedescort  -'^.O       -5,0       -5.0 

If  these  values  are  correct  tvpe  qo :  (qol 

hiqh  oerf  a/c  -50  -50  -50 

hiqh  oerf  a/c  -50.0       -50.0       -50.0 

If  these  values  are    correct     tyoe  qo:  [qol 

normal  mission  0  0  -100  --------  -------- 

normal  mission  0.0         0.0     -100.0 

If  these  values  are    correct     tyoe  qo:  (qol 

Please  enter  reqret/value  matrix  for 
national  influence 

no  action  harassmentshoot  down 

do  not  fly  -50  -100  -70 

do  not  fly  -5  0.0     -1 00.0      -70.0 

If  these  values  are    correct  type  oo:  [qol 

modified  route  -20  -20  -50------  --- — --- 

modifie:!  route  -20.0      -2  0.0      -50.0 

If  these  values  are    correct     tvoe  qo:  lao) 
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aPTied  escort  --iO     0  0   ------__  -«_«-«-_ 

a  r  n  e  d  e  s  c  o  r  t  -  a  0  .  0        0.0        0.0 

I  *     these  values  are  correct  tyoe  ao:  [qo] 

hiah  no  r  f     a/c       -PS  -li)  -10 

hiqhcerfa/c  -25. 0      -10.0      -10.0 

If  these  values  are  correct  tyoe  qo:  [aol 

normal  Tiission  0  -2S  -10  0-------  -------- 

nortial  mission  o.O      -25.0     -10  0.0 

If  these  values  are    correct"  tvoe  oot  [qol 


Please  enter  reqret/value  matrix  for 
reconnaissance  oroo 

no  action  i-iarassmentshoot  ':!own 
do  not  fly  -100  -100  -100 

do  no  t  f 1 y  -10  0.0     -100.0     -100.0 

If  these  values  are  correct  tyoe  oo :  [qol 

modified  route     -70  -50  -50 

modified  route        -7(J.0      -50.0      -50.0 
If  these  values  are    correct  tvpe  qot  [qol 

armed  escort       0  n  o--   --------   -------- 

armedescort  0.0        0.0        0.0 

If  th^se  values  are    correct  tyoe  qo:   [qol 

hioh  oerf  a/c      -10  -10  -10 

hiqhperfa/c         -10.0      -10.0      -10.0 
If  these  values  are    correct  tyoe  go:  [oo] 

normal  mission     -10  -70  -80------   -------- 

normal  mission        -10.0      -70.0      -80.0 
If  these  values  are  correct  tyoe  oo:  (ool 

Please  enter  a  set  of  importance  weights  for  these  criterion 
a  i  re  r  a  f  t  -a  i  re  re  w 
current  information 
national  influence 
reconnaissance  oroo 

wts:     100  10  5  0  30 

Norm:    5«.B    5.^   17.6   17.6 

If  these  values  are    correct     type  oo:  [qol 

i/vhich  Dortion  of  the  model  would  you  like  to  create? 
OPTION 

1  Generate  influence  diaoram 

2  Generate  value/reqret  model 
5     Add  Baysian  indicators 

^     Return  to  main  menu 
Enter  desired  ootion:^ 
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Please  se1(=»ct  one  of  the  followina  ootions 
OPTiori 

1  Disr)1=!v     result's 

2  Eo  i  t     Tio^e  1 

3  Sensitivity 
'4              Load     m  o  -1  e  1 

5  Creatf^     or     gdrj     tQ     nnoriel 

6  Save    TO'iel 

7  New  infomation 

8  Eno  session 
Enter  desired  ontionri 


The  followinq  disolays  are  available 
OPTION 

1  Exnected  value 

2  CoTibineT  value 

3  Event  likelihood 
'4     Values 

5     Value  weiohts 
b     Influence  diaqrarr 
7     Return  to  main  menu 
Enter  desired  option:! 

Exoected  value/rearet 


do  not  fly 

modifier     rou  ^  e 
a  rnned    escort 
hiqh    cerf     a/c 
nor-nal     Tiission 
Please    cress     return 


-5S.a 
-23.8 

-ao.  1 

to  continue. 


The  follo«inq  disolays  ar<^    available 
OPTIO'^J 

1  Expected  value 

2  CoTibinec")  value 

3  Event  liwelih  ood 
^     Va 1 ues 

5  Value  weights 

6  Influence  diaoram 

7  Return  to  main  menu 
Enter  desirei  ootion:2 

Comb  1 ned  Value 

no  action  h a ra s smen t shoo t  down 

do  not  fiv            -3?.^      -ai.2  -35.9 

modified  route        -17.6      -25.9  -5'i.7 

armed  escort           -7. a       -9.1  -20. « 

hiqh  oerf  a/c          -9.1       -9.4  -15.3 

normal  mission          -1.8       -51,5  -96.5 
Please  press  return  to  continue. 
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The  follo'/uino  Hisol^vs  are  availeole 

OPT  TON 

1  Exnecteo  value 

?  Comhinp'i  valu^ 

3  Event  1i<:plihood 

'J  Values 

5  Value  we  i  oh  t  s 

o  Inflijence  rjiaoram 

7  Ret'urn  to  main  rnenu 
Enter  desired  ootion:7 


select  one  o^  the  followinq  options 

DiSD)av  results 
Edit  Ti  o  a  e  1 
Sensitivity 
Load  Todel 

Create  or  add  to  .Todel 
Save  TO "Je  1 
New  infor-nation 
End  session 
desired  ootion:^ 

one  of  the  followinq: 

Event  orot'ability  thresholds 

value  weiaht  thresholds 
Return  to  to  in  menu 


Please 
OPTION 

1 

? 

3 

ii 

5 
o 
7 

8 
Enter 

Select 
OPTIOlvi 

1 

?_ 

3 
Enter  desired  ootionrl 

'/Ihich    event     outco^rie     likelihood    should    be    varien? 
OPT  \{^H 

1     no  action 

?     ha  rassmen t 

5     shoot  down 
Enter  desired  ootion:! 


Exoected  Value  when 
Probability  of  no  action 


1  s : 


do  not  fly 
modified  route 
arTied  escort 
hiah  oerf  a/c 
normal  mission 


0  10  2  0  30  ao  50  60  70  80  ^0  100 
-39  -38  -37  -37  -3^  -35  -3^  -3'4  -33  -3^  -31 
-32  -31  -29  -27  -26  -2^  -23  -21  -20  -IB  -17 
-11  -11  -10*-10*  -"5*  -9*  -8*  -8*  -7*  -7  -6 
-10*-10*-10*-10*-1 0  -10  -10  -10  -«  -9  -9 
-a2  -38  -3^  -50  -2b    -22  -18  -la  -10   -b*  -2* 


Please  oress  return  to  continue. 


Select  one  of  tf^e  folio wina: 
nPTIO'j 

1  Event  orobability  thresholds 

2  Value  weight  thresholds 


^2 


5     Return  to  main  menu 
Enter  desired  ootion:3 

Please  select  one  of  the  followina  ooticns 
n  P  T  T  n  M 


OPTION 
I 

3 
a 
5 
6 
7 
8 


OisDlay  results 

Edit  T  o  n  e 1 

Sensitivity 

Load  T  o  a  e 1 

Create  or  add  to  model 

Save  model 

^Jew  information 

Fnd  session 


Enter  desired  oDtion:6 


Ple-^se  select  one  o^  t'ne  following  ootions 
OPTION 

1  Disolav  results 

2.  Edit  model 

3  Sens  i  t  i  V  i  t  y 

U  Load  model 

5  Create  or  add  to  model 

6  Save  model 

7  "Jew  information 
^  End  session 


Enter  desired  option: 
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APPEf-;OlX    R    -    SA'VipLE    SCEMAPI05 

ci;ban   blockade   sce^jario 

On  25  ^i^rch  198  0,  ohotoargnhs  receiver  from  rnilitary 
surveillance  aircraft  showed  the  existence  of  new 
construction  alnnq  the  south»^rn  coast  of  Cuoa.  Intelligence 
analysts  believe  th^  construction  to  be  a  new  major  Soviet 
nuclear  suhmgrine  suooort  oase.  however,  they  are  undecirjed 
as  to  wnether  it  will  be  caoable  of  resunclyinq  only  nuclear 
fuel  or  botn  nuclear  fuel  and  nuclear  SLH  '^'  m  i  s  s  i  1  e  s  . 

Additional  intelliaence  from  other  sources  has  revealed 
the  cresence  of  hundreds  of  new  technicians  in  the  area.  In 
resconse  to  A-nerican  inquiries,  the  Soviets  have  stated  that 
j-hey  are  merely  neloina  the  Cubans  build  a  new  merchant 
oort.  They  claim  the  technicians  will  leave  as  soon  as  the 
Dort  is  comoleted. 

v^ost  US  ooservers  are  hiohly  concerned  over  these  new 
develonments.  They  feel  the  oresense  of  a  new  oort,  caoaole 
ot  nuclear  retrofit  for  Soviet  submarines,  has  a  serious 
imoact  on  US  national  security.  No  lonqer  would  the  Soviet 
submarines  be  reouired  to  return  to  home  ports.  They  could 
conceivably  oatrol  near  our  shores  for  unlimited  periods  of 
time.  If  a  nuclear  var  ever  did  break  out,  a  missile 
resuDoly  base  so  near-by  could  be  devastating. 

Analysts  oelieve  the  new  construction  is  caused  by  the 
oossibilitv  of  non-ratification  of  the  new  SALT  Treaty. 
They  feel  if  the  Treaty  is  not  ratified,   the   Soviets   will 
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have  a  jumo  in  arT. s  escalation  anrj  the  US  will  have  no  oower 
to  interfpra.  The  =inalvsts  also  Delieve  that  if  the  Treaty 
is  ratifieif  the  construction  will  auickly  cease. 

On  the  'T'orninq  of  2  7  ^''arch/  three  Soviet  shios  were 
rjetectec)  sailino  towards  Cuba.  Reconnaissance  has  revealed 
one  to  oe  a  freiahterr  one  an  arrred  escort  destrover^  and 
the  last  an  escort  submarine.  Intelligence  sources  in  the 
Soviet  Union  have  sai"i  the  freighter  is  carryina  the  first 
suonly  o^  nuclear  fuel  and  -nay  oossibly  be  also  carrying 
SLR>^'  s. 

Fears  that  the  Soviets  may  oe  trying  to  guic*<ly 
establish  a  strongholri  in  Cuba  oromnteo  the  President  to 
order  a  tnree  ship  Task  Force  to  intercept  the  Soviet 
freighter,  fhe  mission  of  the  Task'  Force  is  to  determine  if 
nuclear  suoolies  of  ar^y  tyoe  are  on  board/  ana  if  so*  to 
detain  it  awaiting  further  instructions. 

The  Soviets  immediately  resoonded  by  statina  they  would 
absolutely  deny  any  boarding  of  the  freighter  and  any  such 
action  would  be  net  by  'military  retaliation.  Intelligence 
has  also  revealed  the  ordering  of  at  least  ten  Soviet 
submarines   into   the  area    and  the  rapid  augmentation  of  its 

Atlantic  Force. 

The  President  has  decided  he  can  not  allow  nuclear 
supplies  to  reach  Cuba  and  he  feels  a  blockade  of  some  tyoe 
must  be  placed  around  Cuba.  He  immediately  olaced  the 
Atlantic  Fleet  on  alert  and  ordered  them  to  begin  steaming 
towards  Cuba.   In  addition,  he  also  placed  the  Mediterranean 
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Fleet  on  alert. 

Tne  tyce  =\nd  5i?e  o'^  blockade  has  not  vet  been 
deterTineci.  The  President  has  asked  you  for  your 
recomTema^'ion,  1he  President  scecifically  wants  a 
recoTiTenda  t  i  on  as  to  whether  a  total  olockacJe  of  all 
shiDoinqr  a  nnodified  blockaoe  of  soTie  sort  allowing  merchant 
shiDS  carryina  non-military  carao  to  pasSf  or  a  roving 
oatrol  which  .^oul'i  only  detain  ships  carrying  military 
hardware. 

0 Derations  oersonnel  have  stated  it  would  reguire  the 
entire  (Atlantic  Fleet  ano  halt  of  the  '^'editerranean  Fleet  to 
invoke  a  total  blockade.  A  modified  blockade  would  reguire 
only  the  Atlantic  Fleet  while  the  roving  oatrol  would  only 
neec:i  half  of  the  ^^tlantic  Fleet  resources. 

Intelliaence  analysts  are  unsure  what  the  Soviet 
intentions  will  be  concernina  the  situation.  They  believe 
thpir  ootions  to  be  one  of  three: 

1.  An  attack  of  the  blockading  forces. 

2.  An  attemot  to  run  through  the  blockade  without 
initiatina  any  aaoression*  or 

3.  SubTnission  to  the  blockade.  This  means  the 
Soviets  may  relent  to  having  some  shios  searched 
but  may  havp  others  return  directly  home. 

They  agrees  howeverr  that  two  upcoming  events  will 
help   to   cietermine   their  motives.   The  impendina   vote   on 
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fnp  SALT  Treatv  and  final  determination  of  exactly 
wnat  tvop  of  construction  is  heina  aone  on  the  coast  ara  the 
kev  issues  in  assessina  their  intentions.  In  adaitioof 
they  feel  ♦■i^e  S'-'>LT  Treaty  vote  will  he  effected  by  noth  US 
oublic  ooinion  concernina  the  Treaty  and  current  world 
ODinion  concerninq  the  situation.  The  UM  Security  council 
is  scheduled  to  rreet  in  two  days  to  discuss  the  si'iuation. 

The  analysts  have  assessed  the  oronahility  of  US  oublic 
ooinion  DeinT  oro-S-^LT  as  70%  and  anti-S/^LT  as  being  30%, 
/lorld  ooinion  is  ass'=ssed  as  havinq  a  ^0%  chance  of  being 
oro-US»  iO'i  oro-Soviet/  and  30%  of  oeing  neutral.  The 
analysts  also  currently  aaree  to  a  60%  likelihooo  of  the 
construction  beino  to  suooort  full  nuclear  resuooly^  30%  for 
only  ^iaval  resuonly  (non-ar-Tannents)/  and  10%  as  a  new 
cornme  re  i  a  1  do  r  t  . 

The  items  tne  President  wants  considered  in  the 
selection  of  a  best  course  of  action  are  (in  order  of 
i  moor  t  ance  )  : 

1 )  ' ! ational  security.  The  president  feels  he  can 
not  allow  the  Soviets  to  establish  a  suooly  base 
in  Cuba  , 

2)  Safety  of  US  .'Nationals  currently  in  Cuba.  The 
US  has  recently  beaun  exportino  numerous 
commercial  products  to  Cuba  in  an  effort  to 
noost  a  sagging  economy.  At  present/  it  is 
estimated  there  are  at  least  2000   Ampricans   in 
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Cuf:'a  . 

3)  I'S  oublic  ooinion.  This  is  an  election  year  and 
analysts  feel  there  will  be  oublic  outcry  it  ^e 
force  the  Soviets  into  a  military  confrontation* 
but  the  runlic  will  be  impressed  by  strong 
measures  which  force  the  Soviets  into  accepting 
our  demands. 

^)  Cuban  reaction.  The  US  has  only  recently  begun 
trade  with  Cuba  ana  is  anxious  to  continue  this 
if  at  all  possible.  Any  tyoe  of  blockade 
imposed  around  Cuba  will  have  a  serious  impact 
on  these  trade  agreements. 

5)  Aorld  opinion.  Any  confrontation  between  the  US 
and  U3SP  is  undesirable.  If  the  US  is 
successful  with  the  blockade*  world  opinion  will 
be  high.  However/  if  we  provoke  aaaression  by 
our  actions/  we  will  lose  additional  world 
support  which  is  desperately  oeing  souaht  by  the 
Presi  dent . 
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AIR  RtCOMwaiSSAi-jCE  SCE^'^APIO 

On  S  '"^rcn  I'^BO/  an  unar-ne'J  '^'ohawk  reconnaissance 
aircraft  aborted  its  Tiission  oven  Korea.  It  returned  to  its 
base  where  t^e  c^ew  reoorted  an  interceotion  oy  a 
fiqhter-tyoe  aircraft  from  ivJorth  Korea.  The  crew  of  the 
unarmed  '^ohawk-  had  not  stayed  around  long  enouah  to 
determine  whether  the  intercepter  was  there  to  harass^ 
s^ootf  icienrify,  or  had  just  "haooened  along"  while  on  a 
routin'=>  trainino  fliaht.  Another  reconnaissance  flight  is 
scheduled  for  tomorrow  ana  the  commander  needs  a 
recommendation  concerning  future  fliahts. 

After  furtner  discussion,  the  commander  desires  the 
following  oDtions  evaluated  in  light  of  what  has  haooened 
and  what  the  intelligence  staff  conclude  are  North  Korea's 
intentions: 

1.  Do  not  fly  into  the  area    anymore. 

2.  Flv  into  the  area  on  a  modifiec:i  route.  This 
modifieo  route  would  be  less  sensitive  but  also 
less  oroductive  in  collection  ootential. 

5.  Fly  into  the  same  area  but  now  with  an  armed 
escort  . 

a.        Fly    into    the    area  with   h i ah -oe r f o r mane e 

aircraft.  The  aircraft  would  be  less  vulnerable 
but  would  not  collect  data  as  well  as  the  normal 
aircraft. 
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5.   f^lv   a   nor^nal    '"ission;   do  no*"  c^  anae     ^nythinq 
from  previous  fliahts. 


jhe  ccT^franaer  is  mostly  concerned  with  the  oossiole 
less  of  the  aircraft  and  crew  durino  the  mission.  However, 
he  is  also  concerned  witS  the  loss  of  prestige  or  political 
influence  if  fliohts  were  stoooert  o  <"  altereci.  The  loss  of 
information  ana  the  cossihle  imoact  n  the  reconnaissance 
oro'^ram  in  otn<»r  areas  are  also  imoortant  factors. 

The  intelliaence  section  believes  it  has  assessed  the 
possible  intentions  of  the  Morth  ^orean  Government  on  future 
j  flights.  Sase'i  on  oast  situations  arin  current  capabilities/ 
thev  feel  there  is  onlv  a  slim  chance  (10 '4)  the  Koreans  will 
try  to  shoot  future  aircraft  down,  a  m. o'lerate  chance  (30%) 
they  will  harass,  but  orobably  will  do  nothina  (60%). 
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iPPENOrx  C  -  SA^"PLE  QlltST  IOMNAIf?tS 

QUEST  lO-iriAI^Je  Fiu;)  TF4M  -'Ef-BEPS  WITHOUT  UPINT 

NA'-E: 

RAijK  : 

TEAM  tf  : 

SCENARIO: 

WnAT  .vAS  YOUR  REC  O^-iMENDE  0  ACTIn^■? 

'."jHY  DID  YUU  RECUN'^'E'lP  THAT  ACTIO^J? 

PLEASE  Ai\S.-.'ER  THE  F  OLLCn  I'lG  QUESTIONS  AS  INDICATED. 

1.  Do  vou  feel  this  scenario  was  realistic?   If  no  t     nlease 
QOTiment  in  the  soace  orov/ided. 

a ,  Yes 

b .  No 


2.     /iere    you    anle    •"o     ilentify    any    non-essential     inforTiation 
r)  urine    your    evaluation    of     the    coijrses    of     action?        If     yes^ 
what     was    ttiat     information? 

a.  Yes 

b.  No 


"^ ,       Vie  re    you  able  ♦'o  assess  fhe  criticality  o  '^     the  factors 
which  contributen  to  the  choice  of  the  recommencied  course 
of  action?   If  yes ,     which  factors  were  the  most  critical? 

a .  Yes 

b.  ^'o 


^.     Do    you  feel  confident  that  your  recommendation  was  tne 
"best"/  based  on  the  information  provided?   If  no  r     why  not? 

a ,  Ves 

b.  No 
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QUEST  lUfiNi  r'-?t    FO^    TEA-'    '-iE"^ieFPS    .'J  I  T  H    OP  KjT 

iMAMt  : 

RAf-)K  : 

TEm^'    ^     : 

SCENAPIG: 

WHAT    ''JA3    YQiJR    RECOf.^--iEMOED    ACTIUi\J? 

I    inHY    DIO    you    PECOvmEMD    TnAT    ACTIOM? 

PLEASE     Ar^S.'^t-K     TMe     FULLO/jI'jG    QUESTIOiJS     AS     INDICATED. 

1.     Do    vcu     fee]     t^  ^  9,    scenario    w^s     realistic?       If    no,     olease 
co^'^ent     in    trie    soace    orovided. 

a.  Yes 

b,  iJO 


? ,     '.■■Jere    ycj    aole    to     irj^nt-ify    any    non-essential      infor'^^ation 
during    your    ev-3lua*"ion    of     the    courses    of     action?       Tf     yeS/ 
what     was     t'^at     infor-nation? 

a.       Yas 

h.       Mo 


3 .   i-'.'  ere  yoLJ  able  to  assess  the  criticality  of  the  factors 
which  contributed  to  th^  choice  o  ^     the  recc^nnended  course 
of  action?   If  veSf  which  factors  were  the  rrost  critical? 

a.  Yes 

b.  No 


^ ,    Do  vou  feel  confident  that  your  recorr.menciation  was  the 
"best"»  based  on  the  inforrpation  provided?   If  no/  why  not? 

a .  Yes 

b.  No 


loa 


"  5.  Did  I'lPI'^lT  nres<=nt  the  evaluat-ed  data  in  a  .manner  which 
wa<5  easily  un:)ers'"ood/   If  no/  which  aisnlavs  were  difficult 
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to  understand? 

a.  res 

b ,  f^'  o 


6.   Did  nPI\iT  disclay  its  results  Quickly  enouqh? 
which  disolavs  were  slow? 

a.   Yes 

b  .   -  J  o 


I  f  not , 


7.  "as  it  easy  tn  make  corrections  i 
comment  on  your  oronlenis. 

a.  Yes 

b.  No 


the  model?   If  not/ 


8.   Did  OPT'JT  recover  well  from  any  errors  you  micjht  have 
made?   If  not/  what  was  the  result? 

a.   Yes 

b  .  U  0 

C,        \i/A     -     I     rlio     not     make     any     errors. 
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QUtSTTQ^jNAl^E    FIJP    TEA'^i    -^lEN'BEws    AFTEP    CG^'PLETT^JG    HOTH    SCENARIOS 
M  a '/  F  : 
p  A  .N|  K  : 

TEA-!  :i      : 

PLEASE  AMSwER  THE  FOLLn/;lMG  QUESTIOriS  AS  INDICATED. 

1.  To  wf^at  extent  din  OPIf'JT  aid  in  your  assessment  O'^ 
what  event  was  cioin:T  to  Haonen? 

a.  Confused 

b .  WO  H  e 1 n 

c.  Peii^^orceo    own     ideas 

d.  Clarified 

e.  EnliThtened 

f.  Had  no  neei  to  assess  an  event 

2,  To  ^^at  extent  did  OPP^'T  aid  in  your  decision  process 
for  sel^ctina  a  course  of  action? 

a.  Innored 

b.  Relied  upon  it 

c.  Addecj  confidence 

d.  ;"ias  confusinq 

5.        Do     you     tnink     tnat     OPI'^.'T     helped     to    speed    un     the    decision 
Tia^cinq    Process? 

a.  Yes 

b.  Mo 

'4.  Do  you  think  you  would  have  performed  better  with  or 
without  OPI^lT? 

a  .   ■.^(  i  t  h 

b .   i';  i  t  H  o  u  t 


lOa 


5.   Do  vou  feel  that  OPI^'iT  coul':!  be  irroroved  uoon? 
in  '.^hic'i  arc,^S? 


I  f  yes 


'3  . 

h. 


Ye: 


■JO 


6.  Do  VOU  fe'^l  decision  'r'a^ers     shoulo  use  CPRiT^  or  should 

it  be  left  to  technical  exoerts? 

a.  Decision  '^ak'^rs 

b.  Technical  exoerts 


7 .  '.  hat  do  vou  cerceive  as  oeina  tne  areatest  advantage  in 
usina  nPIlT? 


8.   '/^hat  do  you  nerceive  as  heina  the  areatest  disadvantage 
in  usinq  OPI^JT? 


9.  '.'Joul'i  vo'j  use  QPI'lT  in  the  future  if  it  was  made 
available  to  vou?   Pl'^'ase  coTTient. 
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